Obstacle avoidance during target tracking

ABSTRACT

A method for controlling a movable object includes obtaining current location information of an obstacle while the movable object tracks a target, and determining whether a location of the obstacle corresponds to a reactive region relative to the movable object based on the current location information of the obstacle. In response to determining that the location of the obstacle corresponds to the reactive region, one or more movement characteristics of the movable object is adjusted in a reactive manner to prevent the movable object from colliding with the obstacle. In response to determining that the location of the obstacle does not correspond to the reactive region, the one or more movement characteristics of the movable object is adjusted in a proactive manner to maintain a distance between the movable object and the obstacle to be larger than a predefined distance.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of International Application No.PCT/CN2016/074821, filed on Feb. 29, 2016, the entire contents of whichare incorporated herein by reference.

TECHNICAL FIELD

The disclosed embodiments relate generally to target tracking and moreparticularly, but not exclusively, to obstacle avoidance duringtracking.

BACKGROUND

Movable objects such as unmanned aerial vehicles (UAVs) can be used forperforming surveillance, reconnaissance, and exploration tasks formilitary and civilian applications. A movable object may carry a payloadconfigured to perform a specific function, such as capturing images ofthe surrounding environment or tracking a specific target. For example,a movable object may track an object moving along the ground or throughthe air. Movement control information for controlling a movable objectis typically received by the movable object from a remote device and/ordetermined by the movable object.

When an obstacle is detected between a movable object and a target,additional movement control information may be generated for the movableobject to avoid the obstacle while tracking the target simultaneously.

SUMMARY

There is a need for systems and methods for obstacle avoidance duringtarget tracking. Such systems and methods optionally complement orreplace conventional methods for obstacle avoidance.

In accordance with some embodiments, a method for controlling a movableobject includes obtaining current location information of an obstaclewhile the movable object tracks a target. The method determines, basedon the current location information of the obstacle, whether a locationof the obstacle corresponds to a reactive region relative to the movableobject. In response to determining that the location of the obstaclecorresponds to the reactive region, one or more movement characteristicsof the movable object are adjusted in a reactive manner such thatcollision of the movable object with the obstacle is avoided. Inresponse to determining that the location of the obstacle does notcorrespond to the reactive region, one or more movement characteristicsof the movable object are adjusted in a proactive manner such that adistance between the movable object and the obstacle exceeds a firstpredefined distance.

In accordance with some embodiments, a system for controlling a movableobject comprises one or more processors; memory; and one or moreprograms, wherein the one or more programs are stored in the memory andconfigured to be executed by the one or more processors, the one or moreprograms including instructions for: obtaining current locationinformation of an obstacle while the movable object tracks a target;determining, based on the current location information of the obstacle,whether a location of the obstacle corresponds to a reactive regionrelative to the movable object; in response to determining that thelocation of the obstacle corresponds to the reactive region, adjustingone or more movement characteristics of the movable object in a reactivemanner such that collision of the movable object with the obstacle isavoided; and, in response to determining that the location of theobstacle does not correspond to the reactive region, adjusting one ormore movement characteristics of the movable object in a proactivemanner such that a distance between the movable object and the obstacleexceeds a first predefined distance.

In accordance with some embodiments, a non-transitory computer readablestorage medium stores one or more programs, the one or more programscomprising instructions, which when executed by a movable object, causethe movable object to: obtain current location information of anobstacle while the movable object tracks a target; determine, based onthe current location information of the obstacle, whether a location ofthe obstacle corresponds to a reactive region relative to the movableobject; in response to determining that the location of the obstaclecorresponds to the reactive region, adjust one or more movementcharacteristics of the movable object in a reactive manner such thatcollision of the movable object with the obstacle is avoided; and, inresponse to determining that the location of the obstacle does notcorrespond to the reactive region, adjust one or more movementcharacteristics of the movable object in a proactive manner such that adistance between the movable object and the obstacle exceeds a firstpredefined distance.

In accordance with some embodiments, an unmanned aerial vehicle (UAV)comprises: a propulsion system and one or more sensors. The UAV isconfigured to: obtain, using the one or more sensors, current locationinformation of an obstacle while the UAV tracks a target; determine,based on the current location information of the obstacle, whether alocation of the obstacle corresponds to a reactive region relative tothe UAV; in response to determining that the location of the obstaclecorresponds to the reactive region, adjust one or more movementcharacteristics of the UAV in a reactive manner such that collision ofthe UAV with the obstacle is avoided; and, in response to determiningthat the location of the obstacle does not correspond to the reactiveregion, adjust one or more movement characteristics of the UAV in aproactive manner such that a distance between the UAV and the obstacleexceeds a first predefined distance.

In accordance with some embodiments, a method for controlling a movableobject, the method comprises: obtaining current location information ofan obstacle while the movable object tracks a target; determining, basedon the current location information of the obstacle, whether a locationof the obstacle corresponds to a reactive region relative to the movableobject; in response to determining that the location of the obstaclecorresponds to the reactive region: adjusting one or more movementcharacteristics of the movable object; updating targeting informationbased on a distance between the obstacle and the movable object andsending the updated targeting information to a control unit, wherein thecontrol unit is configured to update a displayed user interface inaccordance with the updated targeting information.

In accordance with some embodiments, a system for controlling a movableobject, the system comprises one or more processors; memory; and one ormore programs, wherein the one or more programs are stored in the memoryand configured to be executed by the one or more processors, the one ormore programs including instructions for: obtaining current locationinformation of an obstacle while the movable object tracks a target;determining, based on the current location information of the obstacle,whether a location of the obstacle corresponds to a reactive regionrelative to the movable object; in response to determining that thelocation of the obstacle corresponds to the reactive region, adjustingone or more movement characteristics of the movable object; updatingtargeting information based on a distance between the obstacle and themovable object; and sending the updated targeting information to acontrol unit, wherein the control unit is configured to update adisplayed user interface in accordance with the updated targetinginformation.

In accordance with some embodiments, a non-transitory computer readablestorage medium stores one or more programs, the one or more programscomprising instructions, which when executed by a movable object, causethe movable object to: obtain current location information of anobstacle while the movable object tracks a target; determine, based onthe current location information of the obstacle, whether a location ofthe obstacle corresponds to a reactive region relative to the movableobject; in response to determining that the location of the obstaclecorresponds to the reactive region, adjust one or more movementcharacteristics of the movable object; update targeting informationbased on a distance between the obstacle and the movable object; andsend the updated targeting information to a control unit, wherein thecontrol unit is configured to update a displayed user interface inaccordance with the updated targeting information.

In accordance with some embodiments, an unmanned aerial vehicle (UAV)comprises a propulsion system and one or more sensors. The UAV isconfigured to: obtain, using the one or more sensors, current locationinformation of an obstacle while the movable object tracks a target;determine, based on the current location information of the obstacle,whether a location of the obstacle corresponds to a reactive regionrelative to the movable object; in response to determining that thelocation of the obstacle corresponds to the reactive region, adjust oneor more movement characteristics of the movable object; update targetinginformation based on a distance between the obstacle and the movableobject; and send the updated targeting information to a control unit,wherein the control unit is configured to update a displayed userinterface in accordance with the updated targeting information.

In accordance with some embodiments, a method for controlling a movableobject, comprises obtaining current location information of an obstaclewhile the movable object tracks a target; generating a plurality of setsof candidate movement characteristics for the movable object based onthe current location information of the obstacle and a set of currentmovement characteristics of the movable object; selecting, from theplurality of sets of candidate movement characteristics for the movableobject, a set of movement characteristics for the movable object; andadjusting one or more movement characteristics of the movable objectbased on the selected set of movement characteristics for the movableobject.

In accordance with some embodiments, a system for controlling a movableobject, the system comprises one or more processors; memory; and one ormore programs, wherein the one or more programs are stored in the memoryand configured to be executed by the one or more processors, the one ormore programs including instructions for: obtaining current locationinformation of an obstacle while the movable object tracks a target;generating a plurality of sets of candidate movement characteristics forthe movable object based on the current location information of theobstacle and a set of current movement characteristics of the movableobject; selecting, from the plurality of sets of candidate movementcharacteristics for the movable object, a set of movementcharacteristics for the movable object; and adjusting one or moremovement characteristics of the movable object based on the selected setof movement characteristics for the movable object.

In accordance with some embodiments, a non-transitory computer readablestorage medium stores one or more programs, the one or more programscomprising instructions, which when executed by a movable object, causethe movable object to: obtain current location information of anobstacle while the movable object tracks a target; generate a pluralityof sets of candidate movement characteristics for the movable objectbased on the current location information of the obstacle and a set ofcurrent movement characteristics of the movable object; select, from theplurality of sets of candidate movement characteristics for the movableobject, a set of movement characteristics for the movable object; andadjust one or more movement characteristics of the movable object basedon the selected set of movement characteristics for the movable object.

In accordance with some embodiments, an unmanned aerial vehicle (UAV)comprises a propulsion system and one or more sensors. The UAV isconfigured to: obtain, using the one or more sensors, current locationinformation of an obstacle while the movable object tracks a target;generate a plurality of sets of candidate movement characteristics forthe movable object based on the current location information of theobstacle and a set of current movement characteristics of the movableobject; select, from the plurality of sets of candidate movementcharacteristics for the movable object, a set of movementcharacteristics for the movable object; and adjust one or more movementcharacteristics of the movable object based on the selected set ofmovement characteristics for the movable object.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a target tracking system, in accordance with someembodiments.

FIG. 2A illustrates an exemplary movable object in a target trackingsystem, in accordance with some embodiments.

FIG. 2B illustrates an exemplary carrier of a movable object, inaccordance with some embodiments.

FIG. 2C illustrates an exemplary payload of a movable object, inaccordance with some embodiments.

FIG. 3 illustrates an exemplary sensing system of a movable object, inaccordance with some embodiments.

FIG. 4 is a block diagram illustrating an implementation of memory of amovable object, in accordance with some embodiments.

FIG. 5 illustrates an exemplary control unit of a target trackingsystem, in accordance with some embodiments.

FIG. 6 illustrates an exemplary computing device for controlling amovable object, in accordance with some embodiments.

FIG. 7 is a flow diagram illustrating a method for implementing targettracking, in accordance with some embodiments.

FIG. 8 illustrates an exemplary configuration of a movable object,carrier, and payload, in accordance with some embodiments.

FIG. 9 illustrates an exemplary tracking method for maintaining anexpected position of a target, in accordance with some embodiments.

FIG. 10 illustrates an exemplary tracking method for maintaining anexpected size of a target, in accordance with some embodiments.

FIG. 11 illustrates an exemplary process for implementing targettracking, in accordance with some embodiments.

FIG. 12 illustrates an exemplary user interface for selecting and/ortracking a target, in accordance with some embodiments.

FIG. 13 illustrates controlling a movable object to avoid an obstacle,in accordance with some embodiments.

FIG. 14 illustrates adjusting a movement characteristic of a movableobject in a proactive manner, in accordance with some embodiments.

FIG. 15 illustrates adjusting a movement characteristic of a movableobject in a reactive manner in accordance with some embodiments.

FIG. 16 illustrates a reactive region, in accordance with someembodiments.

FIG. 17 illustrates sub-regions of a reactive region, in accordance withsome embodiments.

FIGS. 18A-18B illustrate exemplary adjustments made to a user interfacein response to received adjusted target tracking information, inaccordance with some embodiments.

FIG. 19 illustrates a frame of reference used for adjusting one or moremovement characteristics of a movable object in a proactive manner, inaccordance with some embodiments.

FIG. 20 illustrates sets of candidate movement characteristics fordetermining a (V_(Y), V_(Z)) motion adjustment, in accordance with someembodiments.

FIG. 21 illustrates sets of candidate movement characteristics fordetermining a (V_(X), ω_(Z)) motion adjustment, in accordance with someembodiments.

FIGS. 22A-22B illustrate obstacle size criteria applied to determinewhether a (V_(Y), V_(Z)) motion adjustment or a (V_(X), ω_(Z)) motionadjustment is to be used, in accordance with some embodiments.

FIGS. 23A-23F are a flow diagram illustrating a method for controlling amovable object, in accordance with some embodiments.

FIGS. 24A-24G are a flow diagram illustrating a method for controlling amovable object, in accordance with some embodiments.

FIGS. 25A-25G are a flow diagram illustrating a method for controlling amovable object, in accordance with some embodiments.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea thorough understanding of the various described embodiments. However,it will be apparent to one of ordinary skill in the art that the variousdescribed embodiments may be practiced without these specific details.In other instances, well-known methods, procedures, components,circuits, and networks have not been described in detail so as not tounnecessarily obscure aspects of the embodiments.

The following description uses an unmanned aerial vehicle (UAV) as anexample of a movable object. UAVs include, e.g., fixed-wing aircraftsand rotary-wing aircrafts such as helicopters, quadcopters, and aircrafthaving other numbers and/or configurations of rotors. It will beapparent to those skilled in the art that other types of movable objectsmay be substituted for UAVs as described below in accordance withembodiments of the disclosure.

The present disclosure provides describes techniques related to targettracking by UAVs. In some embodiments, a UAV is configured to receivetarget information from a remote control unit, such as a user-operateddevice. The target information is related to a target to be tracked byan imaging device coupled to the UAV. The target information is used bythe UAV to cause the imaging device to automatically track the target,e.g., to maintain a predetermined position and/or size of the targetwithin one or more images captured by the imaging device. In someembodiments, tracking of the target is performed while the UAV iscontrolled by communications from a control unit, such as communicationsincluding user commands and/or predetermined navigation paths. In someembodiments, the control unit is configured to display images from theimaging device as well as allowing user input related to the targetinformation.

In some embodiments, a user selects a target from an image displayed ona user interface of the control unit. For example, the image isdisplayed and the input is received via a touchscreen of the controlunit. In some embodiments, when the target information is configured,the control unit and/or UAV manage operations associated with targettracking. Managing target tracking operations includes, e.g., adjustingmotion of the UAV, adjusting the carrier and/or adjusting the imagingdevice. For example, the attitude, position, velocity, zoom, and/orother aspects of the UAV and/or imaging device are automaticallyadjusted to ensure that the target is maintained at a designatedposition and/or size within the images captured by the imaging device.In some embodiments, images captured during the tracking process (e.g.,videos or pictures) are streamed to the control unit in real time orsubstantially real time for display, playback, storage, and/or otherpurposes. In this manner, a user is enabled to manage target tracking(e.g., by selecting a target for tracking) without the burden ofmanaging operations involved in piloting the UAV to maintain a view ofthe target.

In accordance with various embodiments described herein, a UAV avoidsobstacles detected while the UAV tracks a target. When an obstacle isdetected, a distance between the UAV and the obstacle is determined. Ifthe obstacle does not pose an immediate threat to the safety of the UAV(e.g., still far from the UAV), movement characteristics of the UAV areadjusted in a proactive manner to maintain a predetermined distancebetween the obstacle and the UAV. Proactive adjustment of movementcharacteristics may include selecting a set of potential motionadjustment options and determining route optimization scores for eachoption. Movement of the UAV is adjusted in accordance with theadjustment option that has the highest route optimization score or atleast a route optimization score above a predefined threshold. If theobstacle may cause an immediate threat to the safety of the UAV (e.g.,very close to the UAV), movement characteristics of the UAV are adjustedin a reactive manner to avoid collision of the UAV with the obstacleranging from reducing acceleration or velocity of the UAV to reversingthe motion of the UAV, depending on how close the UAV is to theobstacle.

FIG. 1 illustrates a target tracking system 100, in accordance withvarious embodiments of the present disclosure. Target tracking system100 includes a movable object 102 and a control unit 104. In someembodiments, target tracking system 100 is used to track target 106.

In some embodiments, target 106 includes natural and/or man-made objectssuch geographical landscapes (e.g., mountains, vegetation, valleys,lakes, and/or rivers), buildings, and/or vehicles (e.g., aircrafts,ships, cars, trucks, buses, vans, and/or motorcycles). In someembodiments, the target 106 includes live subjects such as people and/oranimals. In some embodiments, target 106 is moving, e.g., movingrelative to a reference frame (such as the Earth and/or movable object102). In some embodiments, target 106 is static. In some embodiments,target 106 includes an active target system that transmits informationabout target 106, such as the target's GPS location, to movable object102, control unit 104, and/or computing device 126. For example,information is transmitted to movable object 102 via wirelesscommunication from a communication unit of the active target tocommunication system 120 of movable object 102. Active targets include,e.g., friendly vehicles, buildings, and/or troops. In some embodiments,target 106 includes a passive target (e.g., that does not transmitinformation about target 106). Passive targets include, e.g., neutral orhostile vehicles, buildings, and/or troops.

In some embodiments, movable object 102 is configured to communicatewith control unit 104, e.g., via wireless communications 124. Forexample, movable object 102 receives control instructions from controlunit 104 and/or sends data (e.g., data from movable object sensingsystem 122) to control unit 104.

Control instructions include, e.g., navigation instructions forcontrolling navigational parameters of movable object 102 such asposition, orientation, attitude, and/or one or more movementcharacteristics of movable object 102, carrier 108, and/or payload 110.In some embodiments, control instructions include instructions directingmovement of one or more of movement mechanisms 114. For example, controlinstructions are used to control flight of a UAV. In some embodiments,control instructions include information for controlling operations(e.g., movement) of carrier 108. For example, control instructions areused to control an actuation mechanism of carrier 108 so as to causeangular and/or linear movement of payload 110 relative to movable object102. In some embodiments, control instructions are used to adjust one ormore operational parameters for payload 110, such as instructions forcapturing one or more images, capturing video, adjusting a zoom level,powering on or off, adjusting an imaging mode (e.g., capturing stillimages or capturing video), adjusting an image resolution, adjusting afocus, adjusting a viewing angle, adjusting a field of view, adjusting adepth of field, adjusting an exposure time, adjusting a shutter speed,adjusting a lens speed, adjusting an ISO, changing a lens and/or movingpayload 110 (and/or a part of payload 110, such as imaging device 214).In some embodiments, the control instructions are used to controlcommunication system 120, sensing system 122, and/or another componentof movable object 102.

In some embodiments, control instructions from control unit 104 includetarget information, as described further below with regard to FIG. 7.

In some embodiments, movable object 102 is configured to communicatewith computing device 126. For example, movable object 102 receivescontrol instructions from computing device 126 and/or sends data (e.g.,data from movable object sensing system 122) to computing device 126. Insome embodiments, communications from computing device 126 to movableobject 102 are transmitted from computing device 126 to cell tower 130(e.g., via internet 128) and from cell tower 130 to movable object 102(e.g., via RF signals). In some embodiments, a satellite is used in lieuof or in addition to cell tower 130.

In some embodiments, target tracking system includes additional controlunits 104 and/or computing devices 126 configured to communicate withmovable object 102.

FIG. 2A illustrates an exemplary movable object 102 in a target trackingsystem 100, in accordance with some embodiments. In some embodiments,one or more components of movable object, such as processor(s) 116,memory 118, communication system 120, and sensing system 122, areconnected by data connections, such as a control bus 112. A control busoptionally includes circuitry (sometimes called a chipset) thatinterconnects and controls communications between system components.

Movable object 102 typically includes one or more processing units 116,memory 118, one or more network or other communications interfaces 120,sensing system 112, and one or more communication buses 112 forinterconnecting these components. In some embodiments, movable object102 is a UAV. Although movable object 102 is depicted as an aircraft,this depiction is not intended to be limiting, and any suitable type ofmovable object can be used.

In some embodiments, movable object 102 includes movement mechanisms 114(e.g., propulsion mechanisms). Although the plural term “movementmechanisms” is used herein for convenience of reference, “movementmechanisms 114” refers to a single movement mechanism (e.g., a singlepropeller) or multiple movement mechanisms (e.g., multiple rotors).Movement mechanisms 114 include one or more movement mechanism typessuch as rotors, propellers, blades, engines, motors, wheels, axles,magnets, nozzles, animals, and/or human beings. Movement mechanisms 114are coupled to movable object 102 at, e.g., the top, bottom, front,back, and/or sides. In some embodiments movement mechanisms 114 of asingle movable object 102 include multiple movement mechanisms eachhaving the same type. In some embodiments, movement mechanisms 114 of asingle movable object 102 include multiple movement mechanisms havingdifferent movement mechanism types. Movement mechanisms 114 are coupledto movable object 102 (or vice-versa) using any suitable means, such assupport elements (e.g., drive shafts) or other actuating elements (e.g.,actuators 132). For example, an actuator 132 receives control signalsfrom processor(s) 116 (e.g., via control bus 112) that activates theactuator to cause movement of a movement mechanism 114. For example,processor(s) 116 include an electronic speed controller that providescontrol signals to actuators 134.

In some embodiments, the movement mechanisms 114 enable movable object102 to take off vertically from a surface or land vertically on asurface without requiring any horizontal movement of movable object 102(e.g., without traveling down a runway). In some embodiments, movementmechanisms 114 are operable to permit movable object 102 to hover in theair at a specified position and/or orientation. In some embodiments, oneor more of the movement mechanisms 114 are controllable independently ofone or more of the other movement mechanisms 114. For example, whenmovable object 102 is a quadcopter, each rotor of the quadcopter iscontrollable independently of the other rotors of the quadcopter. Insome embodiments, multiple movement mechanisms 114 are configured forsimultaneous movement.

In some embodiments, movement mechanisms 114 include multiple rotorsthat provide lift and/or thrust to movable object. The multiple rotorsare actuated to provide, e.g., vertical takeoff, vertical landing, andhovering capabilities to movable object 102. In some embodiments, one ormore of the rotors spin in a clockwise direction, while one or more ofthe rotors spin in a counterclockwise direction. For example, the numberof clockwise rotors is equal to the number of counterclockwise rotors.In some embodiments, the rotation rate of each of the rotors isindependently variable, e.g., for controlling the lift and/or thrustproduced by each rotor, and thereby adjusting the spatial disposition,velocity, and/or acceleration of movable object 102 (e.g., with respectto up to three degrees of translation and/or up to three degrees ofrotation).

In some embodiments, carrier 108 is coupled to movable object 102. Apayload 110 is coupled to carrier 108. In some embodiments, carrier 108includes one or more mechanisms that enable payload 110 to move relativeto movable object 102, as described further with reference to FIG. 2B.In some embodiments, payload 110 is rigidly coupled to movable object102 such that payload 110 remains substantially stationary relative tomovable object 102. For example, carrier 108 is coupled to payload 110such that payload is not movable relative to movable object 102. In someembodiments, payload 110 is coupled to movable object 102 withoutrequiring carrier 108.

Communication system 120 enables communication with control unit 104and/or computing device 126, e.g., via wireless signals 124. Thecommunication system 120 includes, e.g., transmitters, receivers, and/ortransceivers for wireless communication. In some embodiments, thecommunication is one-way communication, such that data is transmittedonly from movable object 102 to control unit 104, or vice-versa. In someembodiments, communication is two-way communication, such that data istransmitted in both directions between movable object 102 and controlunit 104.

In some embodiments, movable object 102 communicates with computingdevice 126. In some embodiments, movable object 102, control unit 104,and/or the remote device are connected to the Internet or othertelecommunications network, e.g., such that data generated by movableobject 102, control unit 104, and/or computing device 126 is transmittedto a server for data storage and/or data retrieval (e.g., for display bya website).

In some embodiments, sensing system 122 of movable object 102 includesone or more sensors, as described further with reference to FIG. 3. Insome embodiments, movable object 102 and/or control unit 104 use sensingdata generated by sensors of sensing system 122 to determine informationsuch as a position of movable object 102, an orientation of movableobject 102, movement characteristics of movable object 102 (e.g.,angular velocity, angular acceleration, translational velocity,translational acceleration and/or direction of motion along one or moreaxes), proximity of movable object 102 to potential obstacles, weatherconditions, locations of geographical features and/or locations ofmanmade structures.

FIG. 2B illustrates an exemplary carrier 108 in a target tracking system100, in accordance with embodiments. In some embodiments, carrier 108couples a payload 110 to a movable object 102.

In some embodiments, carrier 108 includes a frame assembly including oneor more frame members 202. In some embodiments, frame member 202 iscoupled with movable object 102 and payload 110. In some embodiments,frame member 202 supports payload 110.

In some embodiments, carrier 108 includes one or more mechanisms, suchas one or more actuators 204, to cause movement of carrier 108 and/orpayload 110. Actuator 204 is, e.g., a motor, such as a hydraulic,pneumatic, electric, thermal, magnetic, and/or mechanical motor. In someembodiments, actuator 204 causes movement of frame member 202. In someembodiments, actuator 204 rotates payload 110 about one or more axes,such as three axes: X axis (“pitch axis”), Z axis (“roll axis”), and Yaxis (“yaw axis”), relative to movable object 102. In some embodiments,actuator 204 translates payload 110 along one or more axes relative tomovable object 102.

In some embodiments, carrier 108 includes one or more carrier sensingsystem 206, e.g., for determining a state of carrier 108 or payload 110.Carrier sensing system 206 includes, e.g., motion sensors (e.g.,accelerometers), rotation sensors (e.g., gyroscopes), potentiometers,and/or inertial sensors. In some embodiments, carrier sensing system 206includes one or more sensors of movable object sensing system 122 asdescribed below with regard to FIG. 3. Sensor data determined by carriersensing system 206 includes, e.g., spatial disposition (e.g., position,orientation, or attitude) and/or movement information such as velocity(e.g., linear or angular velocity) and/or acceleration (e.g., linear orangular acceleration) of carrier 108 and/or payload 110. In someembodiments, sensing data and/or state information calculated from thesensing data are used as feedback data to control the movement of one ormore components (e.g., frame member 202, actuator 204, and/or dampingelement 208) of carrier 108. Carrier sensor 206 is coupled to, e.g.,frame member 202, actuator 204, damping element 208, and/or payload 110.In an embodiment, a carrier sensor 206 (e.g., a potentiometer) measuresmovement of actuator 204 (e.g., the relative positions of a motor rotorand a motor stator) and generates a position signal representative ofthe movement of the actuator 204 (e.g., a position signal representativeof relative positions of the motor rotor and the motor stator). In someembodiments, data generated by a carrier sensor 206 is received byprocessor(s) 116 and/or memory 118 of movable object 102.

In some embodiments, the coupling of carrier 108 to movable object 102includes one or more damping elements 208. Damping elements 208 areconfigured to reduce or eliminate movement of the load (e.g., payload110 and/or carrier 108) caused by movement of movable object 102.Damping elements 208 include, e.g., active damping elements, passivedamping elements, and/or hybrid damping elements having both active andpassive damping characteristics. The motion damped by the dampingelements 208 can include one or more of vibrations, oscillations,shaking, or impacts. Such motions may originate from motions of movableobject that are transmitted to the load. For example, the motion mayinclude vibrations caused by the operation of a propulsion system and/orother components of a movable object 101.

In some embodiments, a damping element 208 provides motion damping byisolating the load from the source of unwanted motion by dissipating orreducing the amount of motion transmitted to the load (e.g., vibrationisolation). In some embodiments, damping element 208 reduces themagnitude (e.g., amplitude) of the motion that would otherwise beexperienced by the load. In some embodiments the motion damping appliedby a damping element 208 is used to stabilize the load, therebyimproving the quality of images captured by the load (e.g., imagecapturing device), as well as reducing the computational complexity ofimage stitching steps required to generate a panoramic image based onthe captured images.

Damping element 208 described herein can be formed from any suitablematerial or combination of materials, including solid, liquid, orgaseous materials. The materials used for the damping elements may becompressible and/or deformable. For example, the damping element 208 ismade of, e.g. sponge, foam, rubber, gel, and the like. For example,damping element 208 includes rubber balls that are substantiallyspherical in shape. The damping element 208 is, e.g., substantiallyspherical, rectangular, and/or cylindrical. In some embodiments, dampingelement 208 includes piezoelectric materials or shape memory materials.In some embodiments, damping elements 208 include one or more mechanicalelements, such as springs, pistons, hydraulics, pneumatics, dashpots,shock absorbers, isolators, and the like. In some embodiments,properties of the damping element 208 are selected so as to provide apredetermined amount of motion damping. In some instances, the dampingelement 208 has viscoelastic properties. The properties of dampingelement 208 are, e.g., isotropic or anisotropic. In some embodiments,damping element 208 provides motion damping equally along all directionsof motion. In some embodiments, damping element 208 provides motiondamping only along a subset of the directions of motion (e.g., along asingle direction of motion). For example, the damping element 208 mayprovide damping primarily along the Y (yaw) axis. In this manner, theillustrated damping element 208 reduces vertical motions.

In some embodiments, carrier 108 includes controller 210. Controller 210includes, e.g., one or more controllers and/or processors. In someembodiments, controller 210 receives instructions from processor(s) 116of movable object 102. For example, controller 210 is connected toprocessor(s) 116 via control bus 112. In some embodiments, controller210 controls movement of actuator 204, adjusts one or more parameters ofcarrier sensor 206, receives data from carrier sensor 206, and/ortransmits data to processor 116.

FIG. 2C illustrates an exemplary payload 110 in a target tracking system100, in accordance with some embodiments. In some embodiments, payload110 includes a payload sensing system 212 and a controller 218. In someembodiments, payload sensing system 212 includes an imaging device 214,such as a camera. In some embodiments, payload sensing system 212includes one or more sensors of movable object sensing system 122 asdescribed below with regard to FIG. 3.

Payload sensing system 212 generates static sensing data (e.g., a singleimage captured in response to a received instruction) and/or dynamicsensing data (e.g., a series of images captured at a periodic rate, suchas a video). Imaging device 214 includes, e.g., an image sensor 216 todetect light (such as visible light, infrared light, and/or ultravioletlight. In some embodiments, imaging device 214 includes one or moreoptical devices (e.g., lenses) to focus or otherwise alter the lightonto image sensor 216.

In some embodiments, image sensors 216 includes, e.g., semiconductorcharge-coupled devices (CCD), active pixel sensors using complementarymetal-oxide-semiconductor (CMOS) or N-type metal-oxide-semiconductor(NMOS, Live MOS) technologies, or any other types of sensors. Imagesensor 216 and/or imaging device 214 capture, e.g., images and/or imagestreams (e.g., videos). Adjustable parameters of imaging device 214include, e.g., width, height, aspect ratio, pixel count, resolution,quality, imaging mode, focus distance, depth of field, exposure time,shutter speed and/or lens configuration. In some embodiments, imagingdevice 214 is configured to capture high-definition orultra-high-definition videos (e.g., 720p, 1080i, 1080p, 1440p, 2000p,2160p, 2540p, 4000p, 4320p, and so on).

In some embodiments, payload 110 includes controller 218. Controller 218includes, e.g., one or more controllers and/or processors. In someembodiments, controller 218 receives instructions from processor(s) 116of movable object 102. For example, controller 218 is connected toprocessor(s) 116 via control bus 112. In some embodiments, controller218 adjusts one or more parameters of one or more sensors of payloadsensing system 212, receives data from one or more sensors of payloadsensing system 212; and/or transmits data, such as image data from imagesensor 216, to processor 116, memory 118, and/or control unit 104.

In some embodiments, data generated by one or more sensors of payloadsensor system 212 is stored, e.g., by memory 118. In some embodiments,data generated by payload sensor system 212 are transmitted to controlunit 104 (e.g., via communication system 120). For example, video isstreamed from payload 110 (e.g., imaging device 214) to control unit104. In this manner, control unit 104 displays, e.g., real-time (orslightly delayed) video received from imaging device 214.

In some embodiments, adjustment to the orientation, position, attitude,and/or one or more movement characteristics of movable object 102,carrier 108, and/or payload 110 is generated based at least in part onconfigurations (e.g., preset and/or user configured in systemconfiguration 400) of movable object 102, carrier 108, and/or payload110. For example, adjustment that involves rotation around two axes(e.g., yaw and pitch) is achieved solely by corresponding rotation ofmovable object around the two axes if payload 110 including imagingdevice 214 is rigidly coupled to movable object 102 (and hence notmovable relative to movable object 102) and/or payload 110 is coupled tomovable object 102 via a carrier 108 that does not permit relativemovement between imaging device 214 and movable object 102. The sametwo-axis adjustment is achieved by, e.g., combining adjustment to bothmovable object 102 and carrier 108 if carrier 108 permits imaging device214 to rotate around at least one axis relative to movable object 102.In this case, carrier 108 can be controlled to implement the rotationaround one or two of the two axes required for the adjustment andmovable object 120 can be controlled to implement the rotation aroundone or two of the two axes. For example, carrier 108 includes, e.g., aone-axis gimbal that allows imaging device 214 to rotate around one ofthe two axes required for adjustment while the rotation around theremaining axis is achieved by movable object 102. In some embodiments,the same two-axis adjustment is achieved by carrier 108 alone whencarrier 108 permits imaging device 214 to rotate around two or more axesrelative to movable object 102. For example, carrier 108 includes atwo-axis or three-axis gimbal.

FIG. 3 illustrates an exemplary sensing system 122 of a movable object102, in accordance with some embodiments. In some embodiments, one ormore sensors of movable object sensing system 122 are mounted to theexterior, located within, or otherwise coupled to movable object 102. Insome embodiments, one or more sensors of movable object sensing systemare components of carrier sensing system 206 and/or payload sensingsystem 212. Where sensing operations are described as being performed bymovable object sensing system 122 herein, it will be recognized thatsuch operations are optionally performed by carrier sensing system 206and/or payload sensing system 212.

Movable object sensing system 122 generates static sensing data (e.g., asingle image captured in response to a received instruction) and/ordynamic sensing data (e.g., a series of images captured at a periodicrate, such as a video).

In some embodiments, movable object sensing system 122 includes one ormore image sensors 302, such as image sensor 308 (e.g., a leftstereographic image sensor) and/or image sensor 310 (e.g., a rightstereographic image sensor). Image sensors 302 capture, e.g., images,image streams (e.g., videos), stereographic images, and/or stereographicimage streams (e.g., stereographic videos). Image sensors 302 detectlight, such as visible light, infrared light, and/or ultraviolet light.In some embodiments, movable object sensing system 122 includes one ormore optical devices (e.g., lenses) to focus or otherwise alter thelight onto one or more image sensors 302. In some embodiments, imagesensors 302 include, e.g., semiconductor charge-coupled devices (CCD),active pixel sensors using complementary metal-oxide-semiconductor(CMOS) or N-type metal-oxide-semiconductor (NMOS, Live MOS)technologies, or any other types of sensors.

In some embodiments, movable object sensing system 122 includes one ormore audio transducers 304. For example, an audio detection systemincludes audio output transducer 312 (e.g., a speaker), and audio inputtransducer 314 (e.g. a microphone, such as a parabolic microphone). Insome embodiments, microphone and a speaker are used as components of asonar system. In some embodiments, a sonar system is used to detectcurrent location information of an obstacle (e.g., obstacle 1316 shownin FIG. 15).

In some embodiments, movable object sensing system 122 includes one ormore infrared sensors 306. In some embodiments, a distance measurementsystem includes a pair of infrared sensors e.g., infrared sensor 316(such as a left infrared sensor) and infrared sensor 318 (such as aright infrared sensor) or another sensor or sensor pair. The distancemeasurement system is used to, e.g., measure a distance to a target 106and/or an obstacle 1316.

In some embodiments, a system to produce a depth map includes one ormore sensors or sensor pairs of movable object sensing system 122 (suchas left stereographic image sensor 308 and right stereographic imagesensor 310, audio output transducer 312 and audio input transducer 314;and/or left infrared sensor 316 and right infrared sensor 318. In someembodiments, a pair of sensors in a stereo data system (e.g., astereographic imaging system) simultaneously captures data fromdifferent positions. In some embodiments, a depth map is generated by astereo data system using the simultaneously captured data. In someembodiments, a depth map is used for positioning and/or detectionoperations, such as detecting an obstacle 1316, detecting currentlocation information of an obstacle 1316, detecting a target 106, and/ordetecting current location information for a target 106.

In some embodiments, movable object sensing system 122 includes one ormore global positioning system (GPS) sensors, motion sensors (e.g.,accelerometers), rotation sensors (e.g., gyroscopes), inertial sensors,proximity sensors (e.g., infrared sensors) and/or weather sensors (e.g.,pressure sensor, temperature sensor, moisture sensor, and/or windsensor).

In some embodiments, sensing data generated by one or more sensors ofmovable object sensing system 122 and/or information determined usingsensing data from one or more sensors of movable object sensing system122 are transmitted to control unit 104 (e.g., via communication system120). In some embodiments, data generated one or more sensors of movableobject sensing system 122 and/or information determined using sensingdata from one or more sensors of movable object sensing system 122 isstored by memory 118.

FIG. 4 is a block diagram illustrating an implementation of memory 118,in accordance with some embodiments. In some embodiments, one or moreelements illustrated in FIG. 4 are located in control unit 104,computing device 126, and/or another device.

In some embodiments, memory 118 stores a system configuration 400.System configuration 400 includes one or more system settings (e.g., asconfigured by a manufacturer, administrator, and/or user). For example,a constraint on one or more of orientation, position, attitude, and/orone or more movement characteristics of movable object 102, carrier 108,and/or payload 110 is stored as a system setting of system configuration400.

In some embodiments, memory 118 stores a motion control module 402.Motion control module stores, e.g., control instructions, such ascontrol instructions received from control module 104 and/or computingdevice 126. Control instructions are used for, e.g., controllingoperation of movement mechanisms 114, carrier 108, and/or payload 110.

In some embodiments, memory 118 stores a tracking module 404. In someembodiments, tracking module 404 generates tracking information fortarget 106 that is being tracked by movable object 102. In someembodiments, tracking information is generated based on images capturedby imaging device 214 and/or output from image analysis module 406(e.g., after pre-processing and/or processing operations have beenperformed on one or more images). Tracking information generated bytracking module 404 includes, for example, location, size, or othercharacteristics of target 106 within one or more images. In someembodiments, tracking information generated by tracking module 404 istransmitted to control unit 104 and/or computing device 126 (e.g.,augmenting or otherwise combined with images and/or output from imageanalysis module 406). For example, tracking information is transmittedto control unit 104 in response to a request from control unit 104and/or on a periodic basis.

In some embodiments, memory 118 includes an image analysis module 406.Image analysis module 406 performs processing operations on images, suchas images captured by imaging device 214. In some embodiments, imageanalysis module performs pre-processing on raw image data, such asre-sampling to assure the correctness of the image coordinate system,noise reduction, contrast enhancement, and/or scale spacerepresentation. In some embodiments, processing operations performed onimage data (including image data that has been pre-processed) includefeature extraction, image segmentation, data verification, imagerecognition, image registration, and/or image matching. In someembodiments, output from image analysis module 406 after pre-processingand/or processing operations have been performed on one or more imagesis transmitted to control unit 104.

In some embodiments, memory 118 stores target information 408. In someembodiments, target information 408 is received by movable object 102(e.g., via communication system 120) from control unit 104, computingdevice 126, target 106, and/or another movable object.

In some embodiments, target information 408 includes a time value and/orexpiration time indicating a period of time during which the target 106is to be tracked. In some embodiments, target information 408 includes aflag indicating whether a targeting information entry 408 includesspecific target information 412 and/or target type information 410.

In some embodiments, target information 408 includes target typeinformation 410 such as color, texture, pattern, size, shape, and/ordimension. Target type information 410 is, e.g., provided by a user to auser input device, such as a user input device of control unit 104. Insome embodiments, the user may select a pre-existing target pattern ortype (e.g., a black object or a round object with a radius greater orless than a certain value). In some embodiments, user input to providetarget type information includes user selection of one or more targets106 from within one or more images. In some embodiments, features orcharacteristics of the selected targets are extracted and/or generalizedto produce target type information 410, which is used, e.g., to identifytargets with features or characteristics indicated by target typeinformation 410. In some embodiments, feature extraction is performed bycontrol unit 104, processor(s) 116 of movable object 102, and/orcomputing device 126.

In some embodiments, target information 408 includes specific targetinformation 412 for a specific target 106. Specific target information412 includes, e.g., an image of target 106, an initial position (e.g.,location coordinates, such as pixel coordinates within an image) oftarget 106, and/or a size of target 106 within one or more images (e.g.,images captured by imaging device 214 of payload 110). A size of target106 is stored, e.g., as a length (e.g., mm or other length unit), anarea (e.g., mm² or other area unit), a number of pixels in a line (e.g.,indicating a length, width, and/or diameter), a ratio of a length of arepresentation of the target in an image relative to a total imagelength (e.g., a percentage), a ratio of an area of a representation ofthe target in an image relative to a total image area (e.g., apercentage), a number of pixels indicating an area of target 106, and/ora corresponding distance of target 106 from movable object 102 (e.g., anarea of target 106 changes based on a distance of target 106 frommovable object 102).

In some embodiments, target information 408 includes expected targetinformation 414. Expected target information 414 specifies one or morecharacteristics of target 106, such as a size parameter (e.g., area,diameter, length and/or width), position (e.g., relative to an imagecenter and/or image boundary), and/or shape. In some embodiments, one ormore characteristics of target 106 are determined from an image oftarget 106 (e.g., using image analysis techniques on images captured byimaging device 112). For example, one or more characteristics of target106 are determined from an orientation and/or part or all of identifiedboundaries of target 106. In some embodiments, expected targetinformation includes pixel coordinates and/or pixel counts to indicate,e.g., a size parameter, position, and/or shape of a target 106. In someembodiments, one or more characteristics of the expected targetinformation 414 are to be maintained as movable object 102 tracks target106 (e.g., the expected target information 414 are to be maintained asimages of target 106 are captured by imaging device 214). Expectedtarget information 414 is used, e.g., to adjust movable object 102,carrier 108, and/or imaging device 214, e.g., such that the specifiedcharacteristics of target 106 are substantially maintained. In someembodiments, expected target information 414 is determined based on oneor more of target type 410 and/or specific target information 412. Forexample, a size of a target is determined from specific targetinformation 412 (e.g., an image of a target 106) and a valuerepresenting an area of the target is stored as expected targetinformation 414.

In some embodiments, target information 408 (including, e.g., targettype information 410, information for a specific target 412, and/orexpected target information 414) is generated based on user input, suchas input received at user input device 506 of control unit 104.Additionally or alternatively, target information is generated based ondata from sources other than control unit 104. For example, target typeinformation may be based on stored previous images of target 106 (e.g.,images captured by imaging device 214 and stored by memory 118), otherdata stored by memory 118, and/or data from data stores that are remotefrom control unit 104 and/or movable object 102. In some embodiments,targeting information is generated using a computer-generated image oftarget 106.

In some embodiments, target information 408 is used by movable object102 to track target 106. For example, target information 408 is used bytracking module 404. In some embodiments, target information 408 is usedby an image analysis module 406 to identify target 106. In some cases,target identification involves image recognition and/or matchingalgorithms based on, e.g., CAD-like object models, appearance-basedmethods, feature-based methods, and/or genetic algorithms. In someembodiments, target identification includes comparing two or more imagesto determine, extract, and/or match features contained therein.

The above identified modules or programs (i.e., sets of instructions)need not be implemented as separate software programs, procedures ormodules, and thus various subsets of these modules may be combined orotherwise re-arranged in various embodiments. In some embodiments,memory 118 may store a subset of the modules and data structuresidentified above. Furthermore, memory 118 may store additional modulesand data structures not described above. In some embodiments, theprograms, modules, and data structures stored in memory 118, or anon-transitory computer readable storage medium of memory 118, provideinstructions for implementing respective operations in the methodsdescribed below. In some embodiments, some or all of these modules maybe implemented with specialized hardware circuits that subsume part orall of the module functionality. One or more of the above identifiedelements may be executed by one or more processors 116 of movable object102. In some embodiments, one or more of the above identified elementsis executed by one or more processors of a device remote from movableobject 102, such as control unit 104 and/or computing device 126.

FIG. 5 illustrates an exemplary control unit 104 of target trackingsystem 100, in accordance with some embodiments. In some embodiments,control unit 104 communicates with movable object 102 via communicationsystem 120, e.g., to provide control instructions to movable object 102.Although control unit 104 is typically a portable (e.g., handheld)device, control unit 104 need not be portable. In some embodiments,control unit 104 is a dedicated control device (e.g., dedicated tooperation of movable object 102), a laptop computer, a desktop computer,a tablet computer, a gaming system, a wearable device (e.g., glasses,gloves, and/or helmet), a microphone, and/or a combination thereof.

Control unit 104 typically includes one or more processing units 502, acommunication system 510 (e.g., including one or more network or othercommunications interfaces), memory 504, one or more input/output (I/O)interfaces (e.g., display 506 and/or input device 508) and one or morecommunication buses 512 for interconnecting these components.

In some embodiments, a touchscreen display includes display 508 andinput device 506. A touchscreen display optionally uses LCD (liquidcrystal display) technology, LPD (light emitting polymer display)technology, or LED (light emitting diode) technology, although otherdisplay technologies are used in other embodiments. A touchscreendisplay and processor(s) 502 optionally detect contact and any movementor breaking thereof using any of a plurality of touch sensingtechnologies now known or later developed, including but not limited tocapacitive, resistive, infrared, and surface acoustic wave technologies,as well as other proximity sensor arrays or other elements fordetermining one or more points of contact with the touchscreen display.

In some embodiments, input device 506 includes, e.g., one or morejoysticks, switches, knobs, slide switches, buttons, dials, keypads,keyboard, mouse, audio transducers (e.g., microphone for voice controlsystem), motion sensor, and/or gesture controls. In some embodiments, anI/O interface of control unit 104 includes sensors (e.g., GPS sensors,and/or accelerometers), audio output transducers (e.g., speaker), and/orone or more tactile output generators for generating tactile outputs.

In some embodiments, input device 506 receives user input to controlaspects of movable object 102, carrier 108, payload 110, or a componentthereof. Such aspects include, e.g., attitude, position, orientation,velocity, acceleration, navigation, and/or tracking. For example, inputdevice 506 is manually set by a user to one or more positions, each ofthe positions corresponding to a predetermined input for controllingmovable object 102. In some embodiments, input device 506 is manipulatedby a user to input control instructions for controlling the navigationof movable object 102. In some embodiments, input device 506 is used toinput a flight mode for movable object 102, such as auto pilot ornavigation according to a predetermined navigation path.

In some embodiments, input device 506 is used to input a target trackingmode for movable object 102, such as a manual tracking mode or anautomatic tracking mode. In some embodiments, the user controls movableobject 102, e.g., the position, attitude, and/or orientation of movableobject 102, by changing a position of control unit 104 (e.g., by tiltingor otherwise moving control unit 104). For example, a change in aposition of control unit 104 is detected by, e.g., one or more inertialsensors and output of the one or more inertial sensors is used togenerate command data. In some embodiments, input device 506 is used toadjust an operational parameter of the payload, such as a parameter of apayload sensing system 212 (e.g., to adjust a zoom parameter of imagingdevice 214) and/or a position of payload 110 relative to carrier 108and/or movable object 102.

In some embodiments, input device 506 is used to indicate informationabout target 106, e.g., to select a target 106 to track and/or toindicate target type information 412. In some embodiments, input device506 is used for interaction with augmented image data. For example, animage displayed by display 508 includes representations of one or moretargets 106. In some embodiments, representations of the one or moretargets 106 are augmented to indicate identified objects for potentialtracking and/or a target 106 that is currently being tracked.Augmentation includes, for example, a graphical tracking indicator(e.g., a box) adjacent to or surrounding a respective target 106. Insome embodiments, input device 506 is used to select a target 106 totrack or to change from a target 106 being tracked to a different targetfor tracking. In some embodiments, a target 106 is selected when an areacorresponding to a representation of target 106 is selected by e.g., afinger, stylus, mouse, joystick, or other component of input device 506.In some embodiments, specific target information 412 is generated when auser selects a target 106 to track.

The control unit 112 may also be configured to allow a user to entertarget information using any suitable method. In some embodiments, inputdevice 506 receives a selection of a target 106 from one or more images(e.g., video or snapshot) displayed by display 508. For example, inputdevice 506 receives input including a selection performed by a gesturearound target 106 and/or a contact at a location corresponding to target106 in an image. In some embodiments, Computer vision or othertechniques are used to determine a boundary of a target 106. In someembodiments, input received at input device 506 defines a boundary oftarget 106. In some embodiments, multiple targets are simultaneouslyselected. In some embodiments, a selected target is displayed with aselection indicator to indicate that the target is selected fortracking. In some other embodiments, input device 506 receives inputindicating information such as color, texture, shape, dimension, and/orother characteristics associated with a target 106. For example, inputdevice 506 includes a keyboard to receive typed input indicating targetinformation 408.

In some embodiments, a control unit 104 provides an interface thatenables a user to select (e.g., using input device 506) between a manualtracking mode and an automatic tracking mode. When the manual trackingmode is selected, the interface enables the user to select a target 106to track. For example, a user is enabled to manually select arepresentation of a target 106 from an image displayed by display 508 ofcontrol unit 104. Specific target information 412 associated with theselected target 106 is transmitted to movable object 102, e.g., asinitial expected target information.

In some embodiments, when the automatic tracking mode is selected, theuser does not provide input selecting a target 106 to track. In someembodiments, input device 506 receives target type information 410 fromuser input. In some embodiments, movable object 102 uses the target typeinformation 410, e.g., to automatically identify the target 106 to betracked and/or to track the identified target 106.

Typically, manual tracking requires more user control of the tracking ofthe target and less automated processing or computation (e.g., image ortarget recognition) by processor(s) 116 of movable object 102, whileautomatic tracking requires less user control of the tracking processbut more computation performed by processor(s) 116 of movable object 102(e.g., by image analysis module 406). In some embodiments, allocation ofcontrol over the tracking process between the user and the onboardprocessing system is adjusted, e.g., depending on factors such as thesurroundings of movable object 102, motion of movable object 102,altitude of movable object 102, system configuration 400 (e.g., userpreferences), and/or available computing resources (e.g., CPU or memory)of movable object 102, control unit 104, and/or computing device 126.For example, relatively more control is allocated to the user whenmovable object is navigating in a relatively complex environment (e.g.,with numerous buildings or obstacles or indoor) than when movable objectis navigating in a relatively simple environment (e.g., wide open spaceor outdoor). As another example, more control is allocated to the userwhen movable object 102 is at a lower altitude than when movable object102 is at a higher altitude. As a further example, more control isallocated to movable object 102 if movable object is equipped with ahigh-speed processor adapted to perform complex computations relativelyquickly. In some embodiments, the allocation of control over thetracking process between user and movable object 102 is dynamicallyadjusted based on one or more of the factors described herein.

In some embodiments, control unit 104 includes an electronic device(e.g., a portable electronic device) and an input device 506 that is aperipheral device that is communicatively coupled (e.g., via a wirelessand/or wired connection) and/or mechanically coupled to the electronicdevice. For example, control unit 104 includes a portable electronicdevice (e.g., a smartphone) and a remote control device (e.g., astandard remote control with a joystick) coupled to the portableelectronic device. In this example, an application executed by thesmartphone generates control instructions based on input received at theremote control device.

In some embodiments, the display device 508 displays information aboutmovable object 102, carrier 108, and/or payload 110, such as position,attitude, orientation, movement characteristics of movable object 102,and/or distance between movable object 102 and another object (e.g.,target 106 and/or an obstacle). In some embodiments, informationdisplayed by display device 508 includes images captured by imagingdevice 214, tracking data (e.g., a graphical tracking indicator appliedto a representation of target 106, such as a box or other shape aroundtarget 106 shown to indicate that target 106 is currently beingtracked), and/or indications of control data transmitted to movableobject 102. In some embodiments, the images including the representationof target 106 and the graphical tracking indicator are displayed insubstantially real-time as the image data and tracking information arereceived from movable object 102 and/or as the image data is acquired.

The communication system 510 enables communication with communicationsystem 120 of movable object 102, communication system 610 of computingdevice 126, and/or a base station (e.g., computing device 126) via awired or wireless communication connection. In some embodiments, thecommunication system 510 transmits control instructions (e.g.,navigation control instructions, target information, and/or trackinginstructions). In some embodiments, the communication system 510receives data (e.g., tracking data from payload imaging device 214,and/or data from movable object sensing system 122). In someembodiments, control unit 104 receives tracking data (e.g., via wirelesscommunications 124) from movable object 102. Tracking data is used bycontrol unit 104 to, e.g., display target 106 as the target is beingtracked. In some embodiments, data received by control unit 104 includesraw data (e.g., raw sensing data as acquired by one or more sensors)and/or processed data (e.g., raw data as processed by, e.g., trackingmodule 404).

In some embodiments, memory 504 stores instructions for generatingcontrol instructions automatically and/or based on input received viainput device 506. The control instructions include, e.g., controlinstructions for operating movement mechanisms 114 of movable object 102(e.g., to adjust the position, attitude, orientation, and/or movementcharacteristics of movable object 102, such as by providing controlinstructions to actuators 132). In some embodiments, the controlinstructions adjust movement of movable object 102 with up to sixdegrees of freedom. In some embodiments, the control instructions aregenerated to maintain tracking of a target 106 (e.g., to correct adetected deviation of target 106 from expected target information, asdescribed further with regard to FIG. 7). In some embodiments, controlinstructions include instructions for adjusting carrier 108 (e.g.,instructions for adjusting damping element 208, actuator 204, and/or oneor more sensors of carrier sensing system 206 of carrier 108). In someembodiments, control instructions include instructions for adjustingpayload 110 (e.g., instructions for adjusting one or more sensors ofpayload sensing system 212). In some embodiments, control instructionsinclude control instructions for adjusting the operations of one or moresensors of movable object sensing system 122.

In some embodiments, input device 506 receives user input to control oneaspect of movable object 102 (e.g., the zoom of the imaging device 214)while a control application generates the control instructions foradjusting another aspect of movable object 102 (e.g., to control one ormore movement characteristics of movable object 102). The controlapplication includes, e.g., control module 402, tracking module 404and/or a control application of control unit 104 and/or computing device126. For example, input device 506 receives user input to control one ormore movement characteristics of movable object 102 while the controlapplication generates the control instructions for adjusting a parameterof imaging device 214. In this manner, a user is enabled to focus oncontrolling the navigation of movable object without having to provideinput for tracking the target (e.g., tracking is performed automaticallyby the control application).

In some embodiments, allocation of tracking control between user inputreceived at input device 506 and the control application variesdepending on factors such as, e.g., surroundings of movable object 102,motion of movable object 102, altitude of movable object 102, systemconfiguration (e.g., user preferences), and/or available computingresources (e.g., CPU or memory) of movable object 102, control unit 104,and/or computing device 126. For example, relatively more control isallocated to the user when movable object is navigating in a relativelycomplex environment (e.g., with numerous buildings or obstacles orindoor) than when movable object is navigating in a relatively simpleenvironment (e.g., wide open space or outdoor). As another example, morecontrol is allocated to the user when movable object 102 is at a loweraltitude than when movable object 102 is at a higher altitude. As afurther example, more control is allocated to movable object 102 ifmovable object 102 is equipped with a high-speed processor adapted toperform complex computations relatively quickly. In some embodiments,the allocation of control over the tracking process between user andmovable object is dynamically adjusted based on one or more of thefactors described herein.

FIG. 6 illustrates an exemplary computing device 126 for controllingmovable object 102, in accordance with embodiments. Computing device 126is, e.g., a server computer, laptop computer, desktop computer, tablet,or phone. Computing device 126 typically includes one or more processingunits 602, memory 604, communication system 610 and one or morecommunication buses 612 for interconnecting these components. In someembodiments, computing device 126 includes input/output (I/O) interfaces606, e.g., display 614 and/or input device 616.

In some embodiments, computing device 126 is a base station thatcommunicates (e.g., wirelessly) with movable object 102 and/or controlunit 104.

In some embodiments, computing device 126 provides data storage, dataretrieval, and/or data processing operations, e.g., to reduce theprocessing power and/or data storage requirements of movable object 102and/or control unit 104. For example, computing device 126 iscommunicatively connected to a database 614 (e.g., via communication610) and/or computing device 126 includes database 614 (e.g., database614 is connected to communication bus 612).

Communication system 610 includes one or more network or othercommunications interfaces. In some embodiments, computing device 126receives data from movable object 102 (e.g., from one or more sensors ofmovable object sensing system 122) and/or control unit 104. In someembodiments, computing device 126 transmits data to movable object 102and/or control unit 104. For example, computing device provides controlinstructions to movable object 102.

FIG. 7 is a flow diagram illustrating a method 700 for implementingtarget tracking, in accordance with some embodiments. The method 700 isperformed at a device, such as moving object 102, control unit 104and/or computing device 126. For example, instructions for performingthe method 700 are stored in tracking module 404 of memory 118 andexecuted by processor(s) 116.

The device obtains (702) target information 408 for one or more targets106. For example, target information 408 is obtained from memory 118 ofmovable object 102, memory 504 of control unit 104, and/or memory 604 ofcomputing device 126. In some embodiments, target information 408obtained by the device is expected target information 414.

The device identifies (704) a target 106 based on the target information408. For example, the device uses an image captured by imaging device214 and/or one or more sensors of movable object sensing system 122 toidentify target 106. In some embodiments, target 106 is identified usingimage recognition or identification techniques (e.g., by image analysismodule 406).

The device determines (706) initial expected target information of theidentified target 106. For example, the device determines an initialposition of identified target 106 within an initial image captured byimaging device 214 and/or one or more sensors of movable object sensingsystem 122 and stores the initial position as expected targetinformation 414.

The device determines (708) updated target information of the identifiedtarget 106. For example, the device determines an updated position oftarget 106, e.g., as identified within one or more subsequent imagescaptured after the initial image. In some embodiments, the updatedtarget information of the identified target 106 is stored as expectedtarget information 414 (e.g., replacing previous expected targetinformation 414 for target 106).

In some embodiments, the device compares (710) the updated targetinformation determined at operation 708 with expected target information414 (e.g., to determine an extent to which target 106 has deviated fromexpected target information 414). For example, the device determines adeviation of a position of a representation of target 106, e.g., asidentified within the one or more subsequent images captured after theinitial image, from the position of target 106 within the initial image.

In some embodiments, a deviation of target 106 from the expected targetinformation 414 includes a change in the position of target 106. Achange in the position of target 106 is detected by, e.g., comparingcoordinates of a representation of target 106 (e.g., the coordinates ofa center point of target 106) within an image (e.g., within the one ormore subsequent images captured after the initial image) to expectedtarget information 414 (e.g., coordinates of an expected targetposition).

In some embodiments, a deviation of target 106 from the expected targetinformation 414 includes a change in the size of target 106. A change inthe size of target 106 is determined by comparing a size parameter, suchas an area (e.g., in pixels) of a representation of target 106 within animage (e.g., within the one or more subsequent images captured after theinitial image) to expected target information 414 (e.g., the expectedarea of target 106).

The device determines (712) whether deviation of target 106 from theexpected target information 414, e.g., as determined at operation 710,requires corrective adjustment. In some embodiments, tolerance criteriaare applied to determine whether the deviation of the target from theexpected target information 414 requires corrective adjustment. In someembodiments, tolerance criteria are met when updated target information,such as a position of a representation of target 106 within an image, iswithin a predetermined minimum number of pixels of expected targetinformation 414, such as a position of a representation of target 106 ina prior image. For example, tolerance criteria are met when a positionof a representation of target 106 within an image deviates from aposition of target 106 in accordance with expected target information414 by less than a predefined number of pixels. In some embodiments, thetolerance criteria are met when a size parameter of a representation oftarget 106 is above a minimum and/or below a maximum size parameter. Insome embodiments, the tolerance criteria are met when a size parameterof a representation of target 106 deviates from expected targetinformation by less than a predefined amount (e.g., by less than apredefined number of pixels). In some embodiments, tolerance criteriaare defined by, e.g., system parameters (e.g., system configuration400), e.g., preset and/or adjustable parameters (e.g. adjustable by adevice and/or by a user).

When a deviation of target 106 from the expected target information 414requires corrective adjustment (e.g., the tolerance criteria are notmet), flow proceeds to operation 714. When deviation of target 106 fromthe expected target information 414 does not require correctiveadjustment (e.g., the tolerance criteria are met), flow proceeds tooperation 708.

The device generates instructions (714) to substantially correct thedeviation. In this manner, the device, e.g., substantially corrects thedeviation and/or substantially maintains a representation of target 106in accordance with expected target information 414, for example, tofacilitate ongoing tracking of the target 106. In some embodiments,substantially correcting the deviation includes an adjustment to anorientation, position, attitude, and/or one or more movementcharacteristics of movable object 102, carrier 108, and/or payload 110.In some embodiments, instructions to substantially correct the deviationchanging a parameter of imaging device 214 and/or one or more sensors ofmovable object sensing system 122, e.g., changing zoom, focus, or othercharacteristics associated with imaging device 214.

In some embodiments, an adjustment to substantially correct a deviationincludes adjusting a zoom level of imaging device 214 (e.g., if theimaging device supports the zoom level required), by adjustment to oneor more movement characteristics of movable object 102, or by acombination of adjusting a zoom level of imaging device 214 andadjustment to one or more movement characteristics of movable object102. In some embodiments, a control application (e.g., control module402, tracking module 404 and/or a control application of control unit104 and/or computing device 126) determines one or more adjustments. Forexample, if the imaging device 214 does not support a zoom levelrequired to substantially correct a deviation, one or more movementcharacteristics of movable object 102 are adjusted instead of or inaddition to adjusting the zoom level of imaging device 214.

In some embodiments, the adjustment to the orientation, position,attitude, one or more movement characteristics, and/or another operationparameter of movable object 102, carrier 108, and/or payload 110 islimited by one or more constraints imposed by system configuration 400(e.g., as configured by a manufacturer, administrator, or user), bycontrol unit 104 (e.g., user control input received at control unit104), and/or by computing device 126. Examples of constraints includelimits (e.g., maximum and/or minimum limits) for a rotation angle,angular velocity, and/or linear velocity along one or more axes. Forexample, the angular velocity of movable object 102, carrier 108, and/orpayload 110 around an axis is constrained by, e.g., a maximum angularvelocity that is allowed for movable object 102, carrier 108, and/orpayload 110. In some embodiments, the linear velocity of movable object102, carrier 108, and/or payload 110 is constrained by, e.g., a maximumlinear velocity that is allowed for movable object 102, carrier 108,and/or payload 110. In some embodiments, adjustment to the focal lengthof imaging device 214 is constrained by a maximum and/or minimum focallength for imaging device 214.

In some embodiments, in cases where a navigation path of movable object102 is predetermined, to the orientation, position, attitude, and/or oneor more movement characteristics is implemented by carrier 108 and/orpayload 110 without affecting the movement of movable object 102. Thenavigation path of movable object 102 may be predetermined, for example,if a remote user is actively controlling the navigation of movableobject via a control unit or if movable object is navigating (e.g.,autonomously or semi-autonomously) according to a pre-stored navigationpath.

In some embodiments, a warning indicator is provided when an adjustmentto the orientation, position, attitude, and/or one or more movementcharacteristics is limited by a constraint as described above. In someembodiments, a warning indicator includes text, audio (e.g., siren orbeeping sound), images or other visual indicators (e.g., changed userinterface background color and/or flashing light), and/or hapticfeedback. A warning indicator is provided at, e.g., movable object 102,control unit 104, and/or computing device 126.

In some embodiments, the adjustment to the orientation, position,attitude, and/or one or more movement characteristics is performed insubstantially real time as movable object 102 is executing user-providednavigation control instructions or a predetermined flight path.

In some embodiments, the instructions to substantially correct thedeviation are generated using information, such as sensing data acquiredby one or more sensors of movable object sensing system 122 (e.g.,proximity sensor and/or GPS sensor) and/or position informationtransmitted by target 106 (e.g., GPS location).

In some embodiments, determining updated target information for theidentified target (708) is performed periodically (e.g., every 0.01second, 0.1 second, 0.2 second, 0.5 second, or 1 second) and/or inresponse to a received instruction from movable object 102, carrier 108,and/or payload 110.

FIG. 8 illustrates an exemplary configuration 800 of a movable object102, carrier 108, and payload 110, in accordance with embodiments. Theconfiguration 800 is used to illustrate exemplary adjustments to anorientation, position, attitude, and/or one or more movementcharacteristics of movable object 102, carrier 108, and/or payload 110,e.g., as used to track target 106.

In some embodiments, movable object 102 rotates around up to threeorthogonal axes, such as X₁ (pitch) 810, Y₁ (yaw) 808 and Z₁ (roll) 812axes. Rotations around the three axes are referred to herein as pitchrotation 822, yaw rotation 820, and roll rotation 824, respectively.Angular velocities of movable object 102 around the X₁, Y₁, and Z₁ axesare referred to herein as ω_(X1), ω_(Y1), and ω_(Z1), respectively. Insome embodiments, movable object 102 engages in translational movements828, 826, and 830 along the X₁, Y₁, and Z₁ axes, respectively. Linearvelocities of movable object 102 along the X₁, Y₁, and Z₁ axes arereferred to herein as V_(X1), V_(Y1), and V_(Z1), respectively.

In some embodiments, payload 110 is coupled to movable object 102 viacarrier 108. In some embodiments, payload 110 moves relative to movableobject 102 (e.g., payload 110 is caused by actuator 204 of carrier 108to move relative to movable object 102).

In some embodiments, payload 110 moves around and/or along up to threeorthogonal axes, X₂ (pitch) 816, Y₂ (yaw) 814 and Z₂ (roll) 818. The X₂,Y₂, and Z₂ axes are respectively parallel to the X₁, Y₁, and Z₁ axes. Insome embodiments, where payload 110 includes imaging device 214 (e.g.,including an optical module 802), the roll axis Z₂ 818 is substantiallyparallel to an optical path or optical axis for optical module 802. Insome embodiments, optical module 802 is optically coupled to imagesensor 216 (and/or one or more sensors of movable object sensing system122). In some embodiments, carrier 108 causes payload 110 to rotatearound up to three orthogonal axes, X₂ (pitch) 816, Y₂ (yaw) 814 and Z₂(roll) 818, e.g., based on control instructions provided to actuator 204of carrier 108. The rotations around the three axes are referred toherein as the pitch rotation 834, yaw rotation 832, and roll rotation836, respectively. The angular velocities of payload 110 around the X₂,Y₂, and Z₂ axes are referred to herein as ω_(X2), ω_(Y2), and ω_(Z2),respectively. In some embodiments, carrier 108 causes payload 110 toengage in translational movements 840, 838, and 842, along the X2, Y2,and Z2 axes, respectively, relative to movable object 102. The linearvelocity of payload 110 along the X2, Y2, and Z2 axes is referred toherein as V_(X2), V_(Y2), and V_(Z2), respectively.

In some embodiments, the movement of payload 110 may be restricted(e.g., carrier 108 restricts movement of payload 110, e.g., byconstricting movement of actuator 204 and/or by lacking an actuatorcapable of causing a particular movement).

In some embodiments, the movement of payload 110 may be restricted tomovement around and/or along a subset of the three axes X₂, Y₂, and Z₂relative to movable object 102. For example, payload 110 is rotatablearound X₂, Y₂, Z₂ (movements 832, 834, 836) or any combination thereof,payload 110 is not movable along any of the axes (e.g., carrier 108 doesnot permit payload 110 to engage in movements 838, 840, 842). In someembodiments, payload 110 is restricted to rotation around one of the X₂,Y₂, and Z₂ axes. For example, payload 110 is only rotatable about the Y₂axis (e.g., rotation 832). In some embodiments, payload 110 isrestricted to rotation around only two of the X₂, Y₂, and Z₂ axes. Insome embodiments, payload 110 is rotatable around all three of the X₂,Y₂, and Z₂ axes.

In some embodiments, payload 110 is restricted to movement along X₂, Y₂,or Z₂ axis (movements 838, 840, 842), or any combination thereof, andpayload 110 is not rotatable around any of the axes (e.g., carrier 108does not permit payload 110 to engage in movements 832, 834, 836). Insome embodiments, payload 110 is restricted to movement along only oneof the X₂, Y₂, and Z₂ axes. For example, movement of payload 110 isrestricted to movement 840 along the X₂ axis). In some embodiments,payload 110 is restricted to movement along only two of the X₂, Y₂, andZ₂ axes. In some embodiments, payload 110 is movable along all three ofthe X₂, Y₂, and Z₂ axes.

In some embodiments, payload 110 is able to perform both rotational andtranslational movement relative to movable object 102. For example,payload 110 is able to move along and/or rotate around one, two, orthree of the X₂, Y₂, and Z₂ axes.

In some embodiments, payload 110 is coupled to movable object 102directly without a carrier 108 or carrier 108 does not permit payload110 to move relative to movable object 102. In some embodiments, theattitude, position and/or orientation of payload 110 is fixed relativeto movable object 102 in such cases.

In some embodiments, adjustment to attitude, orientation, and/orposition of payload 110 is performed by adjustment to movable object102, carrier 108, and/or payload 110, such as an adjustment to acombination of two or more of movable object 102, carrier 108, and/orpayload 110. For example, a rotation of 60 degrees around a given axis(e.g., yaw axis) for the payload is achieved by a 60-degree rotation bymovable object alone, a 60-degree rotation by the payload relative tomovable object as effectuated by the carrier, or a combination of40-degree rotation by movable object and a 20-degree rotation by thepayload relative to movable object.

In some embodiments, a translational movement for the payload isachieved via adjustment to movable object 102, carrier 108, and/orpayload 110 such as an adjustment to a combination of two or more ofmovable object 102, carrier 108, and/or payload 110. In someembodiments, a desired adjustment is achieved by adjustment to anoperational parameter of the payload, such as an adjustment to a zoomlevel or a focal length of imaging device 214.

FIG. 9 illustrates an exemplary tracking method for maintaining anexpected position of a target 106, in accordance with embodiments. Anexemplary image 900 is e.g. an image captured by imaging device 214.Assume that the image has a width of W pixels and a height of H pixels(where W and H are positive integers). A position within the image isdefined by a pair of coordinates along an axis 901 (along the width ofthe image) and an axis 903 (along the height of the image), where theupper left corner of image has coordinates (0, 0) and the lower rightcorner of the image has coordinates (W, H).

Assume that a representation of target 106, as captured in the image900, is located at position P (u, v) 902, and the expected position ofthe target (e.g., as indicated by expected target information 414) isP₀(u₀, v₀) 904 that is different from P 902. In some embodiments, theexpected position of the target P₀(u₀, v₀) may be near the center of theimage, such that u₀=W/2, and/or v₀=H/2. In some embodiments, theexpected position of the target is located at other locations within theimage (e.g., off-center). In some embodiments, an expected position ofthe target may or may not be the same as an initial position of thetarget (e.g., as determined at 706). Assuming that the deviation ofcurrent position P from the expected target information 414 (e.g., theexpected position P₀) requires corrective adjustment (e.g., asdetermined at 710-712), instructions are generated (e.g., as generatedat operation 714) for an adjustment to bring the target position from Pto close to the expected position P0. In some embodiments, a deviationis expressed as a Δx from u₀, and a Δy from v₀.

In some embodiments, the deviation from the expected target position isused to derive one or more angular velocities for rotating the field ofview of the imaging device around one or more axes. For example,deviation along the axis 901 of the image (e.g., between u and u₀) isused to determine an angular velocity ω_(Y) 910 for rotating the fieldof view of the imaging device 214 around the Y (yaw) axis 906, asfollows:ω_(Y)=α*(u−u ₀), where α∈

(real numbers)  (1)

In some embodiments, the rotation around the Y axis for the field ofview of imaging device 214 is achieved by a rotation of movable object102, a rotation of payload 110 (e.g., via carrier 108) relative tomovable object 102, or a combination of both. In some embodiments,payload 110 is adjusted when adjustment of movable object 102 isinfeasible or otherwise undesirable, for example, when the navigationpath of movable object is predetermined. In the equation (1), α is aconstant that may be predefined and/or calibrated based on theconfiguration of the movable object (e.g., when the rotation is achievedby the movable object), the configuration of the carrier (e.g., when therotation is achieved by the carrier), or both (e.g., when the rotationis achieved by a combination of the movable object and the carrier). Insome embodiments, a is greater than zero (α>0). In other embodiments, amay be no greater than zero (α≤0). In some embodiments, α can be used tomap a calculated pixel value to a corresponding control lever amount orsensitivity for controlling the angular velocity around a certain axis(e.g., yaw axis). In general, the control lever may be used to controlthe angular or linear movement of a controllable object (e.g., movableobject 102 or carrier 108). A greater control lever amount correspondsto greater sensitivity and greater speed (for angular or linearmovement). In some embodiments, the control lever amount or a rangethereof is determined by configuration parameters of the flight controlsystem (e.g. stored by system configuration 400 and/or motion controlmodule 402) for a movable object 102 or configuration parameters of acontrol system for a carrier 108. The upper and lower bounds of therange of the control lever amount may include any arbitrary numbers. Forexample, the range of the control lever amount may be (1000, −1000) forone flight control system and (−1000, 1000) for another flight controlsystem.

For instance, assume that the images have a width of W=1024 pixels and aheight of H=768 pixels. Thus, the size of the images is 1024*768.Further assume that the expected position of the target has a u₀=512.Thus, (u−u₀)∈(−512, 512). Assume that the range of the control leveramount around the yaw axis is (−1000, 1000), then the maximum controllever amount or maximum sensitivity is 1000 and α=1000/512. Thus, thevalue of α can be affected by image resolution or size provided by theimaging device, range of the control lever amount (e.g., around acertain rotation axis), the maximum control lever amount or maximumsensitivity, and/or other factors.

For instance, when the rotation is achieved by rotation of movableobject 102, the Y axis 906 of FIG. 9 corresponds to the Y₁ axis 808 forthe movable object as illustrated in FIG. 8 and the overall angularvelocity of the field of view ω_(Y) is expressed as the angular velocityω_(Y1) for the movable object:ω_(Y)=ω_(Y1)=α₁*(u−u ₀), where α₁∈

  (2)

In the equation (2), α₁ is a constant that is defined based on theconfiguration of the movable object. In some embodiments, α₁ is greaterthan zero (α₁>0). The α₁ can be defined similar to the α discussedabove. For example, the value of α₁ may be defined based on imageresolution or size and/or range of control lever amount for the movableobject (e.g., around the yaw axis).

Similarly, when the rotation is achieved by the rotation of payload 110relative to movable object 102 (e.g., via carrier 108), the Y axis 906of FIG. 9 corresponds to the Y₂ axis 814 for the payload as illustratedin FIG. 8 and the overall angular velocity of the field of view ω_(Y) isexpressed as the angular velocity ω_(Y2) for the payload relative to themovable object:ω_(Y)=ω_(Y2)=α₂*(u−u ₀), where α₂∈

  (3)

In the equation (3), α₂ is a constant that is defined based on theconfiguration of the carrier and/or payload. In some embodiments, α₂ isgreater than zero (α₂>0). The α₂ can be defined similar to the αdiscussed above. For example, the value of α₂ may be defined based onimage resolution or size and/or range of control lever amount for thecarrier 108 (e.g., around the yaw axis).

In general, the angular velocity of the field of view around the Y (yaw)axis 906 can be expressed as a combination of the angular velocityω_(Y1) for the movable object and the angular velocity ω_(Y2) for thepayload relative to the movable object, such as the following:ω_(Y)=ω_(Y3)+ω_(Y2)  (4)

In the equation (4), either ω_(Y1) or ω_(Y2) may be zero.

As illustrated herein, the direction of the rotation around the Y (yaw)axis may depend on the sign of u−u₀. For instance, if the expectedposition is located to the right of the actual position (as illustratedin FIG. 9), then u−u₀<0, and the field of view needs to rotate in acounter-clockwise fashion around the yaw axis 906 (e.g., pan left) inorder to bring the target to the expected position. On the other hand,if the expected position is located to the left of the actual position,then u−u₀>0, and the field of view needs to rotate in a clockwisefashion around the yaw axis 906 (e.g., pan right) in order to bring thetarget to the expected position.

As illustrated herein, the velocity of rotation (e.g., absolute value ofthe angular velocity) around a given axis (e.g., the Y (yaw) axis) maydepend on the distance between the expected and the actual position ofthe target along the axis (i.e., |u−u₀|). The further the distance is,the greater the velocity of rotation. Likewise, the closer the distanceis, the slower the velocity of rotation. When the expected positioncoincides with the position of the target along the axis (e.g., u=u₀),then the velocity of rotation around the axis is zero and the rotationstops.

The method for adjusting the deviation from the expected target positionand the actual target position along the axis 901, as discussed above,can be applied in a similar fashion to correct the deviation of thetarget along a different axis 903. For example, deviation along the axis903 of the image (e.g., between v and v₀) may be used to derive anangular velocity ω_(X) 914 for the field of view of the imaging devicearound the X (pitch) axis 908, as follows:ω_(X)=β*(v−v ₀), where β∈

  (5)

The rotation around the X axis for the field of view of an imagingdevice may be achieved by a rotation of the movable object, a rotationof the payload 110 (e.g. via carrier 108) relative to the movable object102, or a combination of both. Hence, in the equation (5), β is aconstant that may be predefined and/or calibrated based on theconfiguration of the movable object (e.g., when the rotation is achievedby the movable object), the configuration of the carrier (e.g., when therotation is achieved by the carrier), or both (e.g., when the rotationis achieved by a combination of the movable object and the carrier). Insome embodiments, β is greater than zero (β>0). In other embodiments, βmay be no greater than zero (β≤0). In some embodiments, β can be used tomap a calculated pixel value to a corresponding control lever amount forcontrolling the angular velocity around a certain axis (e.g., pitchaxis). In general, the control lever may be used to control the angularor linear movement of a controllable object (e.g., movable object 102 orcarrier 108). Greater control lever amount corresponds to greatersensitivity and greater speed (for angular or linear movement). In someembodiments, the control lever amount or a range thereof may bedetermined by configuration parameters of the flight control system fora movable object 102 or configuration parameters of a carrier controlsystem for a carrier 108. The upper and lower bounds of the range of thecontrol lever amount may include any arbitrary numbers. For example, therange of the control lever amount may be (1000, −1000) for one controlsystem (e.g., flight control system or carrier control system) and(−1000, 1000) for another control system.

For instance, assume that the images have a width of W=1024 pixels and aheight of H=768 pixels. Thus, the size of the images is 1024*768.Further assume that the expected position of the target has a v₀=384.Thus, (v−v₀)∈(−384, 384). Assume that the range of the control leveramount around the pitch axis is (−1000, 1000), then the maximum controllever amount or maximum sensitivity is 1000 and β=1000/384. Thus, thevalue of β can be affected by image resolution or size provided by theimaging device, range of the control lever amount (e.g., around acertain rotation axis), the maximum control lever amount or maximumsensitivity, and/or other factors.

For instance, when the rotation is achieved by rotation of the movableobject, the X axis 908 of FIG. 9 corresponds to the X₁ axis 810 for themovable object as illustrated in FIG. 8 and the angular velocity of thefield of view ω_(X) is expressed as the angular velocity ω_(X1) for themovable object:ω_(X)=ω_(X1)=β₁*(v−v ₀), where β₁∈

  (6)

In the equation (6), β₁ is a constant that is defined based on theconfiguration of the movable object. In some embodiments, β₁ is greaterthan zero (β₁>0). The β₁ can be defined similar to the β discussedabove. For example, the value of β₁ may be defined based on imageresolution or size and/or range of control lever amount for the movableobject (e.g., around the pitch axis).

Similarly, when the rotation is achieved by the rotation of the payloadrelative to the movable object (e.g., via the carrier), the X axis 908of FIG. 9 corresponds to the X₂ axis 816 for the payload as illustratedin FIG. 8 and the angular velocity of the field of view ω_(X) isexpressed as the angular velocity ω_(X2) for the payload relative to themovable object:ω_(X)=ω_(X2)=β₂*(v−v ₀), where β₂∈

  (6)

In the equation (6), β₂ is a constant that is defined based on theconfiguration of the carrier and/or payload. In some embodiments, β₂ isgreater than zero (β₂>0). The β₂ can be defined similar to the βdiscussed above. For example, the value of β₂ may be defined based onimage resolution or size and/or range of control lever amount for themovable object (e.g., around the pitch axis).

In general, the angular velocity of the field of view around the X(pitch) axis 608 can be expressed as a combination of the angularvelocity ω_(X1) for the movable object and the angular velocity ω_(X2)for the payload relative to the movable object, such as the following:ω_(X)=ω_(X1)+ω_(X2)  (7)

In the equation (7), either ω_(X1) or ω_(X2) may be zero.

As illustrated herein, the direction of the rotation around the X (yaw)axis may depend on the sign of v−v₀. For instance, if the expectedposition is located above of the actual position (as illustrated in FIG.9), then v−v₀>0, and the field of view needs to rotate in a clockwisefashion around the pitch axis 908 (e.g., pitch down) in order to bringthe target to the expected position. On the other hand, if the expectedposition is located to below the actual position, then v−v₀<0, and thefield of view needs to rotate in a counter-clockwise fashion around thepitch axis 608 (e.g., pitch up) in order to bring the target to theexpected position.

As illustrated herein, the speed of rotation (e.g., absolute value ofthe angular velocity) depends on the distance between the expected andthe actual position of the target (i.e., |v−v₀|) along a give axis(e.g., the X (pitch) axis). The further the distance is, the greater thespeed of rotation. The closer the distance is, the slower the speed ofrotation. When the expected position coincides with the position of thetarget (e.g., v=v₀), then the speed of rotation is zero and the rotationstops.

In some embodiments, the values of the angular velocities as calculatedabove may be constrained or otherwise modified by various constraints ofthe system. Such constraints may include the maximum and/or minimumspeed that may be achieved by the movable object and/or the imagingdevice, the range of control lever amount or the maximum control leveramount or maximum sensitivity of the control system for the movableobject and/or the carrier, and the like. For example, the rotation speedmay be the minimum of the calculated rotation speed and the maximumspeed allowed.

In some embodiments, warning indicators are provided (e.g., displayed bydisplay 508 or otherwise output by control unit 104 when the calculatedangular velocities need to be modified according to the constraintsdescribed herein. Examples of such warning indicators may includetextual, audio (e.g., siren or beeping sound), visual (e.g., certaincolor of light or flashing light), mechanical, any other suitable typesof signals. Such warning indicators are provided, e.g., directly by themovable object 102, carrier 108, payload 110, or a component thereof. Insome embodiments, warning indicators are provided by the control unit104 (e.g., via the display 508). In some embodiments, control unit 104provides warning indicators based on instructions received from movableobject 102.

FIG. 10 illustrates an exemplary tracking method for maintaining anexpected size of a target 106, in accordance with embodiments. Anexemplary image 1000 is, e.g., an image captured by an imaging device214 carried by movable object 102. Image 1000 includes a representation1002 of target 106. In some embodiments, the current size s of arepresentation 1002 of target 106 within the image 1000 is indicated inpixels (such as calculated as the product of the width of representation1002 and the height of representation 1002). In some embodiments, theexpected target size (e.g., as indicated by expected target information414) is smaller (e.g., the expected target may be represented by 1004and S=s₀) or larger (e.g., the expected target may be represented by1005 and S=s₁) than the current size s. In some embodiments, theexpected target size is a range extending from s₀ to s₁. The expectedtarget size may or may not be the same as an initial size of the target(e.g., as indicated in specific target information 412, such as specifictarget information 412 provided by control unit 104 to movable object102). In some embodiments, when a deviation of a current target size s(e.g., an area of a representation of target 106 within image 1000) fromthe expected target size s₀ or s₁ (or an expected target size thatranges from s₀ to s₁) requires corrective adjustment (e.g., asdetermined at operation 710-712), instructions are generated (e.g., asgenerated at operation 714) for an adjustment, e.g., to reduce thedeviation of the target size from the expected size.

Although display area of the image 1000 and representation 1002 oftarget 106 are shown as rectangles, this is for illustrative purposesonly and is not intended to be limiting. In some embodiments, anexpected size of the target (e.g. stored as expected target information414) is indicated using representations such as a line (e.g., a radiusor other dimension), circle, oval, polygon, sphere, rectangular prism,and/or polyhedron. Likewise, although the expected target size isexpressed in pixels, this is for illustrative purposes only and notintended to be limiting. In some embodiments, the expected size of thetarget (e.g. stored as expected target information 414) is expressed as,e.g., a length (e.g., mm or other length unit), an area (e.g., mm² orother area unit), a ratio of a length of a representation of the targetin an image relative to a total image length (e.g., a percentage), aratio of an area of a representation of the target in an image relativeto a total image area (e.g., a percentage), a number of pixels in a line(e.g., corresponding to a diameter, length, and/or width of target 106),and/or a number of pixels in an area.

In some embodiments, the deviation of target 106 from expected targetinformation 414 (e.g., the expected target size) is used to derive oneor more linear velocities for movable object 102 and/or payload 110along one or more axes. For example, deviation in the target sizebetween a current target size s and the expected target size S (e.g.,S=s₀ or s₁) can be used to determine a linear velocity V for movingmovable object along a Z (roll) axis 1010, as follows:V=δ*(1−s/S), where δ∈

  (8)

In the equation (8), δ is a constant that is defined based on theconfiguration of movable object or any suitable controllable object(e.g., carrier) that may cause the field of view to move toward and/oraway from the target. In some embodiments, δ is greater than zero (δ>0).In other embodiments, δ may be no greater than zero (δ≤0). In someembodiments, δ can be used to map a calculated pixel value to acorresponding control lever amount or sensitivity for controlling thelinear velocity.

In general, V represents the velocity of movable object 102 toward oraway from the target 106. The velocity vector points from the movableobject 102 to the target 106. If the current size s of therepresentation 1002 of target 106 is smaller than the expected size S,then V>0 and movable object moves towards the target so as to increasethe size of the target as captured in the images. On the other hand, ifthe current size s of the target is larger than the expected size S,then V<0 and movable object moves away from the target so as to reducethe size of the target as captured in the images.

For instance, assume that the images have a width of W=1024 pixels and aheight of H=768 pixels. Thus, the size of the images is 1024*768. Assumethat the range of the control lever amount for controlling the linearvelocity is (−1000, 1000). In an exemplary embodiment, δ=−1000 whens/S=3 and δ=1000 when s/S=⅓.

In some embodiments, the values of the velocities as calculated aboveare constrained or otherwise modified by various constraints of thesystem. Such constraints include, e.g., the maximum and/or minimum speedthat may be achieved by movable object and/or the imaging device, themaximum sensitivity of the control system for movable object and/or thecarrier, and the like. For example, the speed for movable object may bethe minimum of the calculated speed and the maximum speed allowed.

Alternatively or additionally, the deviation between the actual targetsize and the expected target size can be used to derive adjustment tothe operational parameters of the imaging device such as a zoom level orfocal length in order to correct the deviation. Such adjustment to theimaging device may be necessary when adjustment to movable object isinfeasible or otherwise undesirable, for example, when the navigationpath of movable object is predetermined. An exemplary focal lengthadjustment F can be expressed as:F=γ*(1−s/S), where γ∈

  (9)

Where γ is a constant that is defined based on the configuration of theimaging device. In some embodiments, γ is greater than zero (γ>0). Inother embodiments, γ is no greater than zero (γ≤0). The value of γ maybe defined based on the types of lenses and/or imaging devices.

If the current size s of the representation 1002 of target 106 issmaller than the expected size S, then F>0 and the focal lengthincreases by |F| so as to increase the size of the target as captured inthe images. On the other hand, if the actual size s of the target islarger than the expected size S, then F<0 and the focal length decreasesby |F| so as to reduce the current size s of the target 106 as capturedin the images. For example, in an embodiment, γ=10. This means that, forexample, when the actual size of the target is double the size of theexpected size S, the focal length should be decreased by 10 mmaccordingly (i.e., F=10*(1−2/1)=−10) and vice versa.

In some embodiments, the adjustment to the operational parameters of theimaging device such as focal length may be constrained or otherwisemodified by various constraints of the system. Such constraints mayinclude, for example, the maximum and/or minimum focal lengths that maybe achieved by the imaging device 214. As an example, assume the focallength range is (20 mm, 58 mm). Further assume that the initial focallength is 40 mm. Then when s>S, the focal length should be decreasedaccording to equation (9); and when s<S, the focal length should beincreased according to equation (9). However, such adjustment is limitedby the lower and upper bounds of the focal length range (e.g., 20 mm to58 mm). In other words, the post-adjustment focal length should be noless than the minimum focal length (e.g., 20 mm) and no more than themaximum focal length (e.g., 58 mm).

As discussed above in FIG. 9, in some embodiments, warning indicatorsare provided (e.g., at control unit 104) when the calculated adjustment(e.g., linear velocity of movable object or focal length) is modifiedaccording to the constraints described herein.

FIG. 11 illustrates an exemplary process 1100 for implementing targettracking, in accordance with some embodiments. The method 1100 isperformed at a device, such as moving object 102, control unit 104and/or computing device 126. For example, instructions for performingthe method 1100 are stored in tracking module 404 of memory 118 andexecuted by processor(s) 116 of movable object 102.

The device obtains (1102) user control instructions such as navigationcontrol instructions, for example, from control unit 104 and/orcomputing device 126. In some embodiments, the navigation controlinstructions are used for controlling navigational parameters of movableobject 102 such as the position, speed, orientation, attitude, or one ormore movement characteristics of movable object 102. In some cases, thenavigation control instructions include instructions for movable object102 to execute pre-stored navigation control instructions (e.g., storedby motion control module 402) such as control instructions correspondingto a predetermined navigation path. The navigation control instructionsare used, e.g., to control movable object to navigate according to auser-specified or pre-stored navigation path.

The device obtains (1104) target information 408, for example, fromcontrol unit 104 and/or computing device 126. In some embodiments, someor all of target information 408 is obtained from memory 118 (e.g., inlieu of receiving targeting information from control unit 104 and/orcomputing device 126). In some embodiments, some or all of targetinformation 408 is obtained from memory 504, memory 604, and/or database614. The target information 408 includes, e.g., specific targetinformation 412, target type information 410, and/or expected targetinformation 414.

In some embodiments, target information 408 is generated at least inpart based on input received at input device 506 of control unit 104. Insome embodiments, target information 408 is generated at least in partusing data from memory 118, memory 504, memory 604, and/or database 614.For example, target type information 410 is derived based on e.g.,stored images (e.g., images previously captured by imaging device 214).

In some embodiments, the device generates instructions (1106) foradjusting an orientation, position, attitude, and/or one or moremovement characteristics of movable object 102, in response to thenavigation control instructions obtained at operation 1102. In someembodiments, the generated instructions are used for navigation ofmovable object 102 according to a user-specified and/or pre-storednavigation path.

In some embodiments, the device generates instructions (1108) foradjusting an orientation, position, attitude, and/or one or moremovement characteristics of movable object 102, carrier 108, and/orpayload 110 to track target 106 according to the target information 408(e.g., using operations discussed with regard to FIG. 7).

FIG. 12 illustrates an exemplary user interface 1200 for selectingand/or tracking a target 106, in accordance with some embodiments. Insome embodiments, user interface 1200 is displayed by a control unit 104and/or a computing device 126. In some embodiments, the user interfaceis displayed by display 508 of control terminal 104. The user interfaceincludes one or more objects, such as objects 1202, 1204, and 1206. Insome embodiments, one or more of objects 1202, 1204, 1206 is arepresentation of a target 106. In some embodiments, user interface 1200displays an image captured by imaging device 214 and the image includesthe one or more objects 1202, 1204, 1206. In some embodiments, graphicaltracking indicator 1208 is displayed in user interface 1200, e.g.,adjacent to or surrounding a tracked target 106 (e.g., object 1202). Insome embodiments, the position of graphical tracking indicator 1208changes as the position of object 1202 changes, e.g., such thatgraphical tracking indicator 1208 remains associated with (e.g.,adjacent to or surrounding) object 1202 while object 1202 is a trackedtarget 106.

In some embodiments, control unit 104 includes one or more input devices506 for receiving user input. In some embodiments, input received byinput devices 506 is used to provide input indicating an object 1202,1204, 1206 with which graphical tracking indicator 1208 is to beassociated. In this way, a user indicates a target 106 to be tracked, inaccordance with some embodiments. In some embodiments, targetinformation 408 is generated based on received input associatinggraphical tracking indicator 1208 with the object 1202 (e.g., todesignate object 1202 as target 106). In some embodiments, user inputreceived at input device 506 to associate a graphical tracking indicator1208 with an object 1202 includes an input gesture received at a pointthat corresponds to an object (e.g., 1202). In some embodiments, aninput gesture is provided by a contact (e.g., by a finger and/or stylus)at display 508 (e.g., a touchscreen display). In some embodiments, aselection of an object 1202 is provided by user-manipulated input device506 (such as a mouse, button, joystick, keyboard, etc.).

FIG. 13 illustrates controlling a movable object 102 to avoid anobstacle, in accordance with some embodiments.

Movable object 102 moves along a path 1302. In some embodiments, path1302 is a predetermined navigation path and/or a path along whichmovable object 102 moves in response to navigation control instructions(e.g., navigation control instructions received from control unit 104and/or computing device 126). In some embodiments, the path 1302 isdetermined at least in part in response to instructions generated fortracking target 106 (e.g., instructions generated as described withrespect to FIG. 7).

Movable object 102 moves along path 1302 from a first position 1304 atan initial time t₀, to a subsequent position 1306 at a second time t₁that is later than t₀, and so on to positions 1308, 1310 at times t₂,t₃, respectively. Movable object 102 is depicted with broken linesherein to indicate a prior location of movable object 102 at a timeprior to a current time (at which movable object 102 is shown with solidlines) or a later location of movable object 102 at a time after acurrent time.

An obstacle 1316 is located on path 1302, such that movable object 102would eventually collide with obstacle 1316 if the movable object 102continued along path 1302 after time t₃. In some embodiments, obstacle1316 is a substantially static object, such as a manmade and/or naturalstructure, e.g., a traffic sign, radio tower, building, bridge, orgeological feature. In some embodiments, obstacle 1316 is a dynamicobject, such as a vehicle, tree, human, animal, or another movableobject (e.g., a UAV).

In some embodiments, movable object 102 is diverted from path 1302 to analternate path 1318, e.g., such that movable object does avoidscollision with obstacle 1316. For example, at time t₅, movable object102 has moved along alternate path 1318, e.g., to avoid collision withobstacle 1316 and/or to maintain a predetermined distance from obstacle1316.

In some embodiments, different approaches (e.g., a “reactive” approachor a “proactive approach”) are taken to controlling movable object 102to avoid collision depending on a distance between movable object 102and obstacle 1316. In some embodiments, a threshold distance used todetermine whether a reactive approach or a proactive approach will beused is referred to as a “reactive region.” A reactive region istypically defined relative to movable object 102. In some embodiments,when obstacle 1316 is beyond a reactive region of movable object 102(e.g., obstacle 1316 is located at a relatively large distance frommovable object 102, such as a distance exceeding 10 meters), one or moremovement characteristics of movable object 102 are adjusted in aproactive manner, as described further below with reference to FIGS. 14and 19-22.

FIG. 14 illustrates adjusting a movement characteristic of movableobject 102 in a proactive manner, in accordance with some embodiments.As movable object 102 tracks target 106, a navigation path 1402 isdetermined for movable object 102 (e.g., instructions are generated formovable object 102, such as by target tracking process 700). In someembodiments, in response to detecting obstacle 1316, one or moremovement characteristics of movable object 102 are adjusted, e.g., suchthat movable object moves along alternate path 1404. For example, one ormore movement characteristics of movable object 102 are adjusted in aproactive manner. In some embodiments, adjusting one or more movementcharacteristics of movable object 102 in a proactive manner includesadjusting one or more movement characteristics of movable object 102such that a distance between movable object 102 and the obstacle exceedsa predefined distance 1406 (e.g., the distance between movable object102 and obstacle 1316 is maintained at and/or or beyond a predefineddistance 1406 as movable object 102 moves relative to obstacle 1316.Predefined distance 1406 is e.g., a distance between 5 and 20 meters,such as 10 meters.

In some embodiments, after movable object 102 moves along alternate path1404, movable object 102 resumes movement along a navigation path 1408for tracking target 106. In some embodiments, movable object 102 trackstarget 106 continuously as one or more movement characteristics ofmovable object 102 are adjusted in a proactive manner. In someembodiments, movable object 102 ceases to track target 106 when one ormore movement characteristics of movable object 102 are adjusted in aproactive manner and/or when obstacle 1316 is detected. In someembodiments, after ceasing to track target 106, movable object 102resumes tracking of target 106 when obstacle 1316 is avoided (forexample, when, motion of movable object 102 is along a vector thatpoints away from obstacle 1316) and/or when obstacle 1316 is no longerdetected.

FIG. 15 illustrates adjusting a movement characteristic of movableobject 102 in a reactive manner, in accordance with some embodiments.For example, a movement characteristic of movable object 102 is adjustedin a reactive manner, e.g., such that a collision between obstacle 1316and movable object 102 is avoided. As movable object 102 tracks target106, a navigation path 1502 is determined for movable object 102 (e.g.,instructions are generated for movable object 102, such as by targettracking process 700). In some embodiments, in response to detectingobstacle 1316, and in response to determining that the location of theobstacle 1316 corresponds to a reactive region (e.g., obstacle 1316 islocated within 10 meters of movable object 102), one or more movementcharacteristics of movable object 102 are adjusted, e.g., such thatmovable object 102 ceases to move or moves along reverse path 1504. Forexample, one or more movement characteristics of movable object 102 areadjusted in a reactive manner. Adjusting one or more movementcharacteristics of movable object 102 in a reactive manner includesadjusting one or more movement characteristics to reduce theacceleration of movable object 102, reduce the velocity of movableobject 102, cease the motion of movable object 102, and/or to reversethe motion of movable object 102. For example, one or more movementcharacteristics of movable object 102 are adjusted such that movableobject 102 moves along an alternate path 1504 (e.g., in a direction thatincreases distance between movable object 102 and obstacle 1316, such asa direction that is opposite to navigation path 1502).

FIG. 16 illustrates a reactive region 1602, in accordance with someembodiments. Reactive region 1602 is a region in which one or moremovement characteristics of movable object 102 are adjusted in areactive manner, e.g., to avoid collision of movable object 102 withobstacle 1316. Typically, reactive region 1602 is a region definedrelative to movable object 102, such a region centered on movable object102 and/or surrounding movable object 102. For example, reactive region1602 is, e.g., a circle or a sphere (e.g., centered on movable object102 and/or a defined point relative to movable object 102, such as acenter of mass of movable object 102). In some embodiments, reactiveregion 1602 extends from movable object 102 in a direction of movementof a movable object 102. For example, reactive region 1602 is, e.g., acone or triangle (e.g., with a vertex on movable object 102 and/or adefined point relative to movable object 102, such as a center of massof movable object 102). In some embodiments, reactive region 1602 isdefined by a radius 1604. In some embodiments, the length of radius1604, is, e.g., a distance between 5 and 20 meters, such as 10 meters.In some embodiments, reactive region 1602 is a distance from movableobject 102 along an axis that is a line defined by movable object 102(and/or a defined point relative to movable object 102) and obstacle1316 (e.g. a point determined from and/or indicated by current locationinformation for obstacle 1316). In some embodiments, reactive region1602 is a predefined distance from movable object 102 along path 1302.

Obstacle 1316 is discovered, e.g., in response to a periodic scan, inresponse to a user- or device-initiated scan, based on navigationinformation, or in response to a determination that obstacle intersectsa path 1302 of movable object 102. In some embodiments, obstacle 1316 isdiscovered using a stored depth map and/or a depth map generated in realtime (using one or more sensors of movable object sensing system 122).Obstacle 1316 is discovered by movable object 102, control unit 104,computing device 126, and/or based on received user input indicating thepresence of an obstacle. In some embodiments, movable object 102,control unit 104, and/or computing device 126 determines, based oncurrent location information for obstacle 1316, whether a location ofobstacle 1316 corresponds to a reactive region 1602. In someembodiments, detecting an obstacle includes obtaining current locationinformation of an obstacle 1316. In some embodiments, current locationinformation is obtained for an obstacle 1316 in response to detection ofthe obstacle.

Obstacle 1316-A is presented as an example of an obstacle 1316 that doesnot correspond to reactive region 1602 (e.g., because obstacle 1316-A isoutside of reactive region 1602). For example, a point, dimension,outline, area, and/or volume of obstacle 1316-A (e.g., as determined byimage analysis module 406) or a point defined with respect to obstacle1316-A (e.g., a centroid of obstacle 1316-A) is partially (e.g., atleast 90%) or fully outside of and/or beyond reactive region 1602. Insome embodiments, in response to a determination that the location ofobstacle 1316-A does not correspond to the reactive region 1602, one ormore movement characteristics of movable object 102 are adjusted in aproactive manner (e.g., such that a distance between the movable object102 and the obstacle 1316-A exceeds a first predefined distance 1406).In some embodiments, first predefined distance 1406 is greater than orequal to the length of radius 1604.

Obstacle 1316-B is presented as an example of an obstacle 1316 thatcorresponds to reactive region 1602. Obstacle 1316-B is shown within(e.g., at least partially within and/or overlapping) reactive region1602. For example, a point, dimension, outline, area, and/or volume ofobstacle 1316-B (e.g., as determined by image analysis module 406) or apoint defined with respect to obstacle 1316-B (e.g., a centroid ofobstacle 1316-B) is within (e.g., at least partially within, such as atleast 10% within, and/or overlapping) reactive region 1602. In someembodiments, in response to a determination that the location ofobstacle 1316-B corresponds to the reactive region 1602, one or moremovement characteristics of movable object 102 are adjusted in areactive manner (e.g., such that a collision between movable object 102and the obstacle 1316-B is avoided).

FIG. 17 illustrates sub-regions of reactive region 1602, in accordancewith some embodiments. In some embodiments, reactive region 1602includes two or more sub-regions. For example, reactive region 1602 asshown in FIG. 17 includes a first sub-region 1702, a second sub-region1704, and a third sub-region 1706.

First sub-region 1702 of reactive region 1602 is, e.g., a region definedby first boundary 1714 (e.g., a sphere with a radius as indicated at1708) and second boundary 1708 (e.g., a sphere with a radius asindicated at 1710), such as a volume between first boundary 1714 andsecond boundary 1716.

Second sub-region 1704 of reactive region 1602 is, e.g., a regiondefined by second boundary 1716 (e.g., a sphere with a radius asindicated at 1710), and third boundary 1718 (e.g., a sphere with aradius as indicated at 1712), such as a volume between second boundary1716 and third boundary 1718.

Third sub-region 1706 of reactive region 1602 is, e.g., a region definedby third boundary 1718, such as a spherical volume inside boundary 1718.

In some embodiments, the lengths of radii 1712, 1710, and 1708 are,e.g., 2 meters, 5 meters, and 10 meters, respectively.

In some embodiments, one or more of first sub-region 1702, secondsub-region 1704, and third sub-region 1706 is defined by a distance frommovable object 102 along an axis that is a line from movable object 102to obstacle 1316. In some embodiments, one or more of first sub-region1702, second sub-region 1704, and third sub-region 1706 is defined byone or more circles within a particular plane, and/or other geometricshapes, volumetric shapes, and or irregular shapes.

In some embodiments, when a determined location of an obstacle 1316corresponds to first sub-region 1702, one or more movementcharacteristics of the movable object 102 are adjusted to, e.g., reducean acceleration of the movable object 102 (such as acceleration in thedirection of obstacle 1316-C). In FIG. 17, obstacle 1316-C correspondsto first sub-region 1702 because obstacle 1316-C is located within(e.g., at least partially within) first sub-region 1702. For example, alocation of obstacle 1316-C corresponds to first sub-region 1702 when apoint, dimension, outline, area, and/or volume of obstacle 1316-C (e.g.,as determined by image analysis module 406) or a point defined withrespect to obstacle 1316-C (e.g., a centroid of obstacle 1316-C) ispartially (e.g., at least 10%) within first sub-region 1702.

In some embodiments, when a determined location of an obstacle 1316corresponds to second sub-region 1704, one or more movementcharacteristics of the movable object 102 are adjusted to, e.g., reducea velocity of the movable object 102 (such as velocity in the directionof obstacle 1316-D). In FIG. 17, obstacle 1316-D corresponds to secondsub-region 1704 (e.g., obstacle 1316-D is located within (e.g., at leastpartially within) second sub-region 1702). For example, a location ofobstacle 1316-D corresponds to second sub-region 1704 when a point,dimension, outline, area, and/or volume of obstacle 1316-D (e.g., asdetermined by image analysis module 406) or a point defined with respectto obstacle 1316-D (e.g., a centroid of obstacle 1316-D) is partially(e.g., at least 10%) within second sub-region 1704.

In some embodiments, when a determined location of an obstacle 1316corresponds to a third sub-region 1706, one or more movementcharacteristics of the movable object 102 are adjusted to, e.g., reversemovement direction of the movable object 102 and/or cease movement ofthe movable object 102. For example movement of movable object 102toward obstacle 1316-E is reversed such that movable object 102 ceasesto move toward obstacle 1316-E and begins to move away from obstacle1316-E. In FIG. 17, obstacle 1316-E corresponds to third sub-region 1706because obstacle 1316-E is located within (e.g., at least partiallywithin) third sub-region 1706. For example, a location of obstacle1316-E corresponds to third sub-region 1706 when a point, dimension,outline, area, and/or volume of obstacle 1316-E (e.g., as determined byimage analysis module 406) or a point defined with respect to obstacle1316-E (e.g., a centroid of obstacle 1316-E) is partially (e.g., atleast 10%) within third sub-region 1706.

FIGS. 18A-18B illustrate exemplary adjustments made to user interface1200 in response to received adjusted target tracking information, inaccordance with some embodiments.

In FIG. 18A, display 508 displays a first state 1800 of a user interface1200 for selecting and/or tracking a target 106, e.g., as described withreference to FIG. 12. For example, in the first state 1800, selectionand/or ongoing tracking of target 106 is indicated by graphical trackingindicator 1208.

In FIG. 18B, display 508 displays a second state 1850 of user interface1200, e.g., a second state 1850 of the user interface 1200 presented inresponse to received adjusted target tracking information. In someembodiments, adjusted target tracking information is received by controlunit 104 from computing device 126 and/or movable object 102. Forexample, updated target tracking is generated in response to determiningthat the location of obstacle 1316 corresponds to the reactive region1602, first sub-region 1702, second sub-region 1704 and/or thirdsub-region 1706. In some embodiments, adjusted target trackinginformation is generated in response to adjusting one or more movementcharacteristics of the movable object 102 in a reactive manner.

In some embodiments, in response to receiving adjusted target trackinginformation (e.g., when one or more movement characteristics of themovable object 102 are updated in a reactive manner) at least one aspectof the appearance of graphical tracking indicator 1208 is changed. Forexample, a color of an outline of graphical tracking indicator 1208 ischanged, part or all of the area of graphical tracking indicator 1208becomes shaded, a color of shading of part or all of graphical trackingindicator 1208 is changed, and/or a size of graphical tracking indicator1208 is changed. In some embodiments, graphical tracking indicatoroverlaps (e.g., partially overlaps) obstacle 1316 and/or target 106. Insome embodiments a shading of at least part of graphical trackingindicator 1208 is transparent (e.g., partially transparent) such thatobstacle 1316 and/or target 106 is visible through the shading ofgraphical tracking indicator 1208. In some embodiments, in response toreceiving adjusted target tracking information, graphical trackingindicator 1208 ceases to be displayed. Changing at least one aspect ofthe appearance of graphical tracking indicator 1208 and/or ceasing todisplay graphical tracking indicator 1208 occurs, e.g., to indicate thattracking of target 106 has been disabled (e.g., temporarily disabled).

In some embodiments, in response to receiving adjusted target trackinginformation (e.g., when one or more movement characteristics of themovable object 102 are adjusted in a reactive manner) a user isprevented from selecting a target 106.

In some embodiments, in response to receiving adjusted target trackinginformation (e.g., when one or more movement characteristics of themovable object 102 are adjusted in a reactive manner), warning indicator1852 is presented. In some embodiments, warning indicator 1852 includestext, images or other visual indicators (e.g., changed user interfacebackground color and/or flashing light) displayed by display 508, audio(e.g., siren or beeping sound) output by control unit 104, and/or hapticfeedback output by control unit 104.

FIG. 19 illustrates a frame of reference used for adjusting one or moremovement characteristics of the movable object 102 in a proactivemanner, in accordance with some embodiments. In some embodiments, aframe of reference 1900 is defined relative to a point corresponding tomovable object 102, such as a center of gravity or central point of adimension (e.g., length, width, or height) an area, or a volume movableobject 102. In some embodiments, movable object 102 as shown in FIG. 19is moving along a path 1302 as shown in FIG. 13. Frame of reference 1900includes velocity vector components V_(X), V_(Y), V_(Z) along an x-axis,y-axis, and z-axis respectively. In some embodiments, V_(x) 1902 isoriented along a vector of movement of movable object 102 (e.g., avector pointing along path 1302, such as a vector along an axis that isa line defined by a point corresponding to movable object 102 and apoint corresponding to target 106). In some embodiments, In someembodiments, V_(Y) 1904 and V_(Z) 1906 are orthogonal to V_(X) 1902.Angular velocity ω_(Z) 1908 (yaw) indicates a velocity of rotation abouta z-axis. In some embodiments, adjusting one or more movementcharacteristics of the movable object in a proactive manner includesadjusting movement characteristics along axes V_(Y) 1904 and axis V_(Z)1906 (e.g., preferentially adjusting movement characteristics along axesV_(Y) 1904 and axis V_(Z) 1906), for example, to allow movable object102 to maintain a predetermined distance from obstacle 1316 whilemaintaining and/or minimally adjusting angular velocity ω_(Z) 1908and/or movement along V_(X) 1902.

FIG. 20 illustrates sets of candidate movement characteristics fordetermining a (V_(Y), V_(Z)) motion adjustment, in accordance with someembodiments. In some embodiments, in response to determining that thelocation of the obstacle does not correspond to reactive region 1602,one or more movement characteristics of the movable object 102 areadjusted in a proactive manner (e.g., such that a distance between themovable object 102 and the obstacle 1316 exceeds a first predefineddistance). In some embodiments, adjusting one or more movementcharacteristics of the movable object 102 in a proactive manner includesdetermining route optimization scores for multiple sets of candidatemovement characteristics. A set of candidate movement characteristicsis, e.g., a set of (V_(Y), V_(Z)) coordinates (e.g., set 2002) in frameof reference 1900. Plot 2000 includes multiple sets (e.g., sets 2002,2004, 2006, and so on) of candidate movement characteristics. In someembodiments, a route optimization score is determined for each set of(V_(Y), V_(Z)) coordinates. In some embodiments, the route optimizationscore is an indication of factors such as predicted amount of timebefore movable object 102 will collide with obstacle 1316, differencesbetween the set of candidate movement characteristics and the set ofcurrent movement characteristics of movable object 102, and/or adistance between movable object 102 and target 106 (e.g., a target beingtracked) at a predetermined future time. In some embodiments, a set of(V_(Y), V_(Z)) coordinates that has a highest route optimization scoreis selected and one or more movement characteristics of movable object102 are adjusted based on the selected set.

For example, sets 2002, 2004, 2006, correspond to (V_(Y), V_(Z))coordinates (24, 21), (26, 21), (28, 21) respectively, where each valueindicates a component of a velocity vector along respective axes V_(Y),V_(Z) (e.g., in meters per second). One or more rules are applied todetermine route optimization scores for each set. For example, routeoptimization scores of 0.4, 0.62, and 0.51 are determined for sets 2002,2004, and 2006, respectively. In this example (in which only three setsare evaluated for ease of description), set 2004 is selected because set2004 has the highest route optimization score. Movement of movableobject 102 is adjusted, e.g., to adjust its (V_(Y), V_(Z)) motion to(26, 21) meters per second.

FIG. 21 illustrates sets of candidate movement characteristics fordetermining a (V_(X), ω_(Z)) motion adjustment, in accordance with someembodiments. In some embodiments, in response to determining that thelocation of the obstacle does not correspond to reactive region 1602,one or more movement characteristics of the movable object 102 areadjusted in a proactive manner (e.g., such that a distance between themovable object 102 and the obstacle 1316 exceeds a first predefineddistance). In some embodiments, (V_(X), ω_(Z)) adjustment criteria areapplied to determine whether a (V_(Y), V_(Z)) adjustment or a (V_(X),ω_(Z)) motion adjustment is to be used. For example, in someembodiments, when a (V_(Y), V_(Z)) adjustment will be insufficient toadjust movement characteristics of the movable object 102 such that adistance between the movable object 102 and the obstacle 1316 exceeds afirst predefined distance, a (V_(X), ω_(Z)) motion adjustment is used.In some embodiments, (V_(X), ω_(Z)) adjustment criteria include adetermination of whether a size of obstacle 1316 exceeds a thresholdsize (e.g., a threshold size that varies depending on motion of movableobject 102 and/or obstacle 1316), such that a (V_(X), ω_(Z)) motionadjustment is used when the size of obstacle 1316 exceeds the thresholdsize. In some embodiments, adjusting one or more movementcharacteristics of the movable object 102 in a proactive manner includesa (V_(Y), V_(Z)) adjustment and a (V_(X), ω_(Z)) motion adjustment.

In some embodiments, adjusting one or more movement characteristics ofthe movable object 102 in a proactive manner includes determining routeoptimization scores for multiple sets of candidate (V_(X), ω_(Z))movement characteristics. A set of candidate movement characteristicsis, e.g., a set of (V_(X), ω_(Z)) coordinates (e.g., set 2102) in frameof reference 1900. Plot 2100 includes multiple sets (e.g., sets 2102,2104, 2106, and so on) of candidate movement characteristics. In someembodiments, a route optimization score is determined for each set of(V_(X), ω_(Z)) coordinates. In some embodiments, the route optimizationscore is an indication of factors such as predicted amount of timebefore movable object 102 will collide with obstacle 1316, differencesbetween the set of candidate movement characteristics and the set ofcurrent movement characteristics of movable object 102, and/or adistance between movable object 102 and target 106 (e.g., a target beingtracked) at a predetermined future time. In some embodiments, a set of(V_(X), ω_(Z)) coordinates that has a highest route optimization scoreis selected and one or more movement characteristics of movable object102 are adjusted based on the selected set.

FIGS. 22A-22B illustrate (V_(X), ω_(Z)) adjustment criteria (e.g.obstacle size criteria) applied to determine whether a (V_(Y), V_(Z))adjustment or a (V_(X), ω_(Z)) motion adjustment is to be used, inaccordance with some embodiments. In some embodiments obstacle sizecriteria are met when a size of obstacle 1316 exceeds a threshold size.In FIG. 22A, obstacle 1316-F exceeds a threshold size. For example, thesize of 1316-F is sufficiently large that a (V_(Y), V_(Z)) adjustmentwill be insufficient, so a (V_(X), ω_(Z)) adjustment is needed. In someembodiments, because obstacle 1316-F exceeds the threshold size,obstacle size criteria are met, and a (V_(X), ω_(Z)) set is selectedfrom plot 2100 for adjusting one or more movement characteristics ofmovable object 102.

In FIG. 22B, obstacle 1316-G does not exceed a threshold size. Forexample, the size of 1316-G is sufficiently small that a (V_(Y), V_(Z))adjustment will be sufficient. In some embodiments, because obstacle1316-G does not exceed the threshold size, (V_(X), ω_(Z)) obstacle sizecriteria are not met, and a (V_(Y), V_(Z)) set is selected from plot2000 for adjusting one or more movement characteristics of movableobject 102.

FIGS. 23A-23F are a flow diagram illustrating a method 2300 forcontrolling a movable object, in accordance with some embodiments. Themethod 2300 is performed at a device, such as moving object 102, controlunit 104 and/or computing device 126. For example, instructions forperforming the method 2300 are stored in motion control module 402 ofmemory 118 and executed by processor(s) 116.

The device obtains (2212) current location information of an obstacle1316 while movable object 102 tracks a target 106. Current locationinformation for the obstacle 1316 includes, e.g., an absolute positionof the obstacle 1316 (such as GPS coordinates of obstacle 1316), arelative position of the obstacle (e.g., a vector from movable object102 to the obstacle 1316), a scalar distance from movable object 102 tothe obstacle 1316, and/or one or more motion attributes of the obstacle,such as a velocity, acceleration and/or direction of movement of theobstacle 1316.

The device determines (2214), based on the current location informationof the obstacle 1316, whether a location of the obstacle 1316corresponds to (e.g., is within) a reactive region 1602 (e.g., reactiveobstacle avoidance region) relative to movable object 102.

In response to determining that the location of the obstacle 1316corresponds to (e.g., is within) the reactive region 1602, the deviceadjusts (2216) one or more movement characteristics of movable object102 in a reactive manner (e.g., reducing an acceleration of movableobject 102 along one or more axes, reducing a velocity of movable object102 along one or more axes, and/or reversing a direction of movement ofmovable object 102) such that collision of movable object 102 with theobstacle 1316 is avoided.

In response to determining that the location of the obstacle 1316 doesnot correspond to (e.g., is not within) the reactive region 1602, thedevice adjusts (2308) one or more movement characteristics of movableobject 102 in a proactive manner such that a distance between movableobject 102 and the obstacle 1316 exceeds a first predefined distance.

In some embodiments, the current location information of the obstacle1316 includes information obtained (2310) using one or more depth maps.In some embodiments, multiple depth maps are used, e.g., to determinemotion attributes of obstacle 1316, such as velocity of obstacle 1316,acceleration of obstacle 1316 and/or direction of movement of obstacle1316 along one or more axes.

In some embodiments, a depth map is a set of points including anindication of a distance to a surface from a viewpoint (e.g., a distanceto a surface that is closest to the viewpoint) corresponding to eachpoint. For example, a depth map is an image in which each pixel includesan indication of a distance to a surface (e.g., the surface nearest tothe viewpoint) from a viewpoint. In some embodiments, the viewpoint ismovable object 102 and/or a part thereof.

In some embodiments, a respective depth map is obtained (2312) using oneor more sensors of movable object sensing system 122. In someembodiments, the at least one sensor is a light sensor (e.g., leftstereographic image sensor 308, right stereographic image sensor 310,left infrared sensor 316, and/or right infrared sensor 318). In someembodiments, the at least one sensor is a pressure sensor (such as asound pressure level sensor, e.g., one or more audio transducers 314).In some embodiments, a depth map is obtained suing a sonar systemincluding audio output transducer 312 and audio input transducer 314.

In some embodiments, the at least one sensor of movable object 102includes (2314) a pair of sensors (left stereographic image sensor 308,right stereographic image sensor 310) for depth mapping.

In some embodiments, the current location information of the obstacle1316 includes (2316) information obtained using at least one sensor ofmovable object 102 (e.g., one or more sensors of movable object sensingsystem 122). In some embodiments, the at least one sensor is used todetermine a distance to an obstacle 1316 (e.g., in lieu of or inaddition to using depth map to determine current location information ofthe obstacle 1316).

In some embodiments, the current location information of the obstacle1316 includes a position (2318) of the obstacle 1316, such as GPScoordinates of the obstacle 1316.

In some embodiments, the current location information includes (2320)one or more movement characteristics (e.g. velocity (e.g., velocityalong one or more axes), acceleration (e.g., acceleration along one ormore axes) and/or a vector indicating a direction of movement of theobstacle 1316.

In some embodiments, the current location information of the obstacle1316 is transmitted (2322) from a computing device 126 to movable object102 (e.g., communicated via a wireless communication channel andreceived by a communication system of movable object).

In some embodiments, the current location information of the obstacle1316 is received (2324) by movable object 102 from a computing device126.

In some embodiments, the reactive region 1602 is defined (2326) at leastin part based on a determined distance from movable object 102 (e.g.,the reactive region 1602 is a circular area and/or spherical volumecorresponding to movable object 102 (e.g., centered on movable object102), and the circular area and/or spherical volume has a radius 1604equal to a predetermined distance from movable object 102. In someembodiments, the determined distance is based on one or more currentmovement characteristics of movable object 102. For example, as thevelocity of movable object 102 increases, the determined distancebetween movable object 102 and the obstacle 1316 increases, e.g.,because movable object 102 requires more time for adjustment to avoidobstacle 1316.

In some embodiments, the determined distance is further based (2328) onone or more current movement characteristics of the obstacle 1316. Forexample, if obstacle 1316 is moving toward movable object 102, thedetermined distance between movable object 102 and the obstacle 1316increases, e.g., because movable object 102 requires more time foradjustment to avoid obstacle 1316.

In some embodiments, in response to determining that the location of theobstacle 1316 does not correspond to the reactive region, the device(2330): selects multiple sets of candidate movement characteristics(e.g., plot 2000, plot 2100) based on the one or more movementcharacteristics of movable object 102; obtains a route optimizationscore for each set of candidate movement characteristics of the multiplesets of candidate movement characteristics in accordance with a firstset of rules; selects the candidate movement characteristics that have ahighest route optimization score; and adjusts the one or more movementcharacteristics of movable object 102 based on the selected candidatemovement characteristics.

In some embodiments, the device predicts (2332), for a set of candidatemovement characteristics, a time at which movable object 102 willcollide with the obstacle 1316; and determines a route optimizationscore for the set of candidate movement characteristics based on adifference between a current time and the predicted time at whichmovable object 102 will collide with the obstacle 1316. For example, aroute optimization score assigned to the set of candidate movementcharacteristics increases as the difference between the current time andthe predicted time increases.

In some embodiments, the predicted amount of time before movable object102 will collide with the obstacle 1316 is determined (2334) using oneor more movement characteristics of the obstacle 1316.

In some embodiments, the route optimization score for a set of candidatemovement characteristics depends (2336), at least in part, upon one ormore differences between the set of candidate movement characteristicsand the set of current movement characteristics of movable object 102.For example, as the differences between a set of candidate movementcharacteristics (V_(Y), V_(Z)) and/or (V_(X), ω_(Z)) and a current(V_(Y), V_(Z)) and/or (V_(X), ω_(Z)) increases (e.g., increases with ahigher ΔV_(Y), ΔV_(Z), ΔV_(X) and/or Δω_(Z)).

In some embodiments, the device predicts (2338), for a set of candidatemovement characteristics, a distance between movable object 102 andtarget 106 at a predetermined future time. A predetermined time is,e.g., a time that is a predetermined amount of time from a current time(e.g., a time between 0.1 and 10 seconds, such as 3 seconds). Thepredetermined time varies based on, e.g., one or more movementcharacteristics of movable object 102 and/or target 106. In someembodiments, the route optimization score for a set of candidatemovement characteristics depends, at least in part, upon the predicteddistance between movable object 102 and the target 106 at thepredetermined future time. For example, as the distance between movableobject 102 and the target 106 decreases, the route optimization scoreincreases.

In some embodiments, a respective set of candidate movementcharacteristics includes (2340) a first movement characteristic in afirst direction that is perpendicular to movement of the movable object102 (e.g., along a path 1302 and/or in a direction of movable object 102as it tracks target 106) and a second movement characteristic in asecond direction that is perpendicular to the movement of movable object102. For example, the first direction is perpendicular to the seconddirection, and both the first direction and the second direction areperpendicular to an axis aligned along a path of movement of movableobject 102.

In some embodiments, a respective set of candidate movementcharacteristics includes (2342) a y-axis movement characteristic V_(Y)and a z-axis movement characteristic V_(Z) (e.g., as described withregard to FIGS. 19 and 20).

In some embodiments, a respective set of candidate movementcharacteristics includes (2344) a movement characteristic in a directionof movement of movable object 102 (e.g., along path 1302 and/or in adirection of movement of movable object 102 as it tracks target 106) andan angular velocity.

In some embodiments, a respective set of candidate movementcharacteristics includes (2346) an x-axis movement characteristic V_(X)and an angular velocity ω_(X) (e.g., as described with regard to FIGS.19 and 21).

In some embodiments, the device determines (2348) a size of the obstacle1316; determines whether the size of the obstacle 1316 meets firstobstacle size criteria, wherein: in response to determining that thesize of the obstacle 1316 meets first obstacle size criteria, themultiple sets of candidate movement characteristics include movement ofmovable object 102 along a y-axis and a z-axis relative to movableobject 102 (e.g., (V_(Y), V_(Z))), and in response to determining thatthe size of the obstacle does not meet first obstacle size criteria, arespective set of candidate movement characteristics of the multiplesets of candidate movement characteristics include: movement of movableobject 102 along an x-axis relative to movable object 102 and angularvelocity of movable object 102 (e.g., (V_(X), ω_(X))).

In some embodiments, in response to determining that the location of theobstacle 1316 corresponds to the reactive region 1602, adjusting the oneor more movement characteristics includes (2350): determining, based onthe current location information of the obstacle 1316, whether thelocation of the obstacle 1316 corresponds to the first sub-region 1702of the reactive region relative to movable object 102; and in responseto determining that the location of the obstacle 1316 corresponds to thefirst sub-region 1702 of the reactive region relative to movable object102, reducing an acceleration (e.g., along one or more axes) of movableobject 102. For example, an acceleration of movable object 102 towardobstacle 1316 is reduced.

In some embodiments, in response to determining that the location of theobstacle 1316 corresponds to the reactive region 1602, adjusting the oneor more movement characteristics includes (2352): determining, based onthe current location information of the obstacle 1316, whether thelocation of the obstacle 1316 corresponds to a second sub-region of thereactive region relative to movable object 102; and in response todetermining that the location of the obstacle 1316 corresponds to thesecond sub-region 1704 of the reactive region 1602 relative to movableobject 102, reducing a velocity (e.g., along one or more axes) ofmovable object 102.

In some embodiments, an area (and/or volume) of the second sub-region1704 of the reactive region relative to movable object 102 is smaller(2354) than an area (and/or volume) of the first sub-region 1702 of thereactive region 1602 relative to movable object 102.

In some embodiments, in response to determining that the location of theobstacle 1316 corresponds to the reactive region 1602, adjusting the oneor more movement characteristics includes (2356): determining, based onthe current location information of the obstacle 1316, whether thelocation of the obstacle 1316 corresponds to a third sub-region 1706 ofthe reactive region 1602 relative to movable object 102; and, inresponse to determining that the location of the obstacle 1316corresponds to the third sub-region 1706 of the reactive region relativeto movable object 102, reversing a direction of movement (e.g., alongone or more axes) of movable object 102.

In some embodiments, an area (and/or volume) of the third sub-region1706 of the reactive region 1602 relative to movable object 102 issmaller (2358) than the area (and/or volume) of the second sub-region1706 of the reactive region relative to movable object 102.

In some embodiments, in response to determining that the location of theobstacle 1316 corresponds to the reactive region 1602, the devicetransmits (2360) updated targeting information (e.g., an indicationand/or instruction that tracking of target 106 is to be suspended) to acontrol unit 104 (e.g., as described with regard to FIGS. 18A-18B).

In some embodiments, the adjusted target tracking information includes(2362) information about the obstacle 1316 (e.g., an image of obstacle1316, one or more movement characteristics of obstacle 1316, and/orcurrent location information of obstacle 1316).

FIGS. 24A-24G are a flow diagram illustrating a method 2400 forcontrolling a movable object 102, in accordance with some embodiments.The method 2400 is performed at a device, such as moving object 102,control unit 104 and/or computing device 126. For example, instructionsfor performing the method 2400 are stored in motion control module 402of memory 118 and executed by processor(s) 116.

The device obtains (2402) current location information of an obstacle1316 while the movable object 102 tracks a target 106.

The device determines (2406), based on the current location informationof the obstacle 1316, whether a location of the obstacle 1316corresponds to a reactive region 1602 relative to the movable object102.

In response to determining that the location of the obstacle 1316corresponds to the reactive region, the device (2408): adjusts one ormore movement characteristics of the movable object 102, adjusts targettracking information based on a distance between the obstacle 1316 andthe movable object 102, and sends the adjusted target trackinginformation to a control unit 104. The control unit 104 is configured toupdate a displayed user interface 1200 in accordance with the adjustedtarget tracking information.

In some embodiments, the distance between the obstacle 1316 and themovable object 102 is determined (2410) based on the current locationinformation of the obstacle 1316.

In some embodiments, the distance between the obstacle 1316 and themovable object 102 is determined (2412) based on output of one or moresensors of movable object sensing system 122.

In some embodiments, adjusting the one or more movement characteristicsof the movable object 102 includes (2414) adjusting the one or moremovement characteristics of the movable object 102 based on: thedistance between the obstacle 1316 and the movable object 102, and oneor more current movement characteristics of the movable object 102.

In some embodiments, adjusting the one or more movement characteristicsof the movable object 102 further includes (2416): determining one ormore movement characteristics of the obstacle 1316. The one or moremovement characteristics of the movable object 102 are adjusted based onmotion of the obstacle 1316.

In some embodiments, the motion of the obstacle 1316 is determined(2418) based on the current location information of the obstacle 1316.

In some embodiments, the motion of the obstacle 1316 is determined(2420) based on output of one or more sensors of movable object sensingsystem 122.

In some embodiments, adjusting the one or more movement characteristicsof the movable object 102 further includes (2422) applying a firstmovement adjustment to movement of the movable object 102 (e.g., towardsthe obstacle 1316) when the distance between the obstacle 1316 and themovable object 102 meets first distance criteria (e.g., when theobstacle 1316 is located within a first region 1702 surrounding themovable object 102). In some embodiments, when the distance between theobstacle 1316 and the movable object 102 meets second distance criteria(e.g., the obstacle 1316 is located within a second region 1704surrounding the movable object 102, the second region 1704 smaller thanthe first region 1702), the device applies a second movement adjustmentto movement of the movable object 102 (e.g., towards the obstacle 1316).

In some embodiments, the first distance criteria are met (2424) when thedistance between the obstacle 1316 and the moving object 120 exceeds afirst distance (e.g., a distance of radius 1710) and the second distancecriteria are met when the distance between the obstacle 1316 and themoving object 102 exceeds a second distance (e.g., a distance of radius1712) that is smaller than the first distance.

In some embodiments, the first distance criteria are met (2426) when thelocation of the obstacle 1316 corresponds to a first sub-region 1702 ofthe reactive region 1602 relative to the movable object 102 and thesecond distance criteria are met when the location of the obstacle 1316corresponds to a second sub-region 1704 of the reactive region 1602relative to the movable object 102. For example, the first distancecriteria are met when obstacle 1316 is located within first sub-region1702 (for example, beyond second sub-region 1704) and the seconddistance criteria are met when the obstacle 1316 is located within asecond sub-region 1704 (e.g., beyond a third sub-region 1706).

In some embodiments, applying the first movement adjustment includes(2428) reducing an acceleration of the movable object. In someembodiments, applying the second movement adjustment includes (2428)reducing a velocity of the movable object.

In some embodiments, adjusting the one or more movement characteristicsof the movable object 102 includes (2430), when the distance between theobstacle 1316 and the movable object 102 meets third distance criteria,applying a third movement adjustment to movement of movable object 102.In some embodiments, the third distance criteria are met when thedistance between the obstacle 1316 and the moving object 102 is a thirddistance that is smaller than the second distance (e.g., the distancebetween the obstacle 1316 and the moving object 102 is a distance lessthan or equal to the distance of radius 1712).

In some embodiments, applying the third movement adjustment includesreversing motion (2432) of the movable object.

In some embodiments, applying the third movement adjustment includesceasing motion (2434) of the movable object.

In some embodiments, adjusting the one or more movement characteristicsof the movable object 102 includes (2436): when the distance between theobstacle 1316 and the movable object 102 meets third distance criteria,applying a third movement adjustment to movement of movable object 102(e.g., reversing a direction of movement of movable object 102), whereinthe third distance criteria are met when the location of the obstacle1316 corresponds to a third sub-region 1706 of the reactive region 1602relative to the movable object 102. For example, the third distancecriteria are met when obstacle 1316 is located within third sub-region1706.

In some embodiments, adjusting the one or more movement characteristicsof the movable object 102 includes (2438): when the distance between theobstacle 1316 and the movable object 102 meets third distance criteria,determining whether a velocity of the movable object 102 meets velocitycriteria; in response to determining that the velocity of the movableobject 102 meets velocity criteria, ceasing motion of the movable object102; and, in response to determining that the velocity of the movableobject 102 does not meet velocity criteria, reversing a direction ofmovement of the movable object 102.

In some embodiments, the velocity criteria are met (2440) when thevelocity of the movable object 102 exceeds a threshold velocity (e.g., athreshold velocity between 5 meters per second and 60 meters per second,such as a threshold velocity of 25 meters per second).

In some embodiments, the adjusted target tracking information includes(2442) information for altering a graphical tracking indicator 1208corresponding to the target 106.

In some embodiments, the information for altering the graphical trackingindicator 1208 corresponding to the target 106 includes (2444)information for changing a color of the graphical tracking indicator1208.

In some embodiments, the information for altering the graphical trackingindicator 1208 corresponding to the target 106 includes (2446)information for ceasing to display the graphical tracking indicator1208.

In some embodiments, the device determines (2448) an updated location ofthe obstacle 1316. For example, the device determines an updatedlocation of obstacle 1316 after altering graphical tracking indicator1208 and/or after adjusting one or more movement characteristics ofmovable object 102. The updated location of obstacle 1316 depends on,e.g., movement of movable object 012 and/or movement of obstacle 1316.In some embodiments, the device determines (2450) whether the updatedlocation of the obstacle 1316 corresponds to the reactive region 1602relative to the movable object 102 (e.g., the device determines whetherthe obstacle is at least partially within reactive region 1602). Inresponse to determining that the updated location of the obstacle 1316does not correspond to the reactive region 1602 relative to the movableobject 102, the device further adjusts (2452) the target trackinginformation. In some embodiments, the device sends the further adjustedtarget tracking information to the control unit 104. In someembodiments, the further adjusted target tracking information includesinformation for ceasing to alter the graphical tracking indicator 1208.For example, when the movable object 102 is no longer at risk of(imminent) collision with obstacle 1316, graphical tracking indicator isre-displayed and/or displayed with its previous color.

In some embodiments, the adjusted target tracking information includes(2454) information for altering (e.g., temporarily altering) a responseto a predefined user input received by the control unit 104.

In some embodiments, the predefined user input includes (2456) controlinput for controlling at least one movement characteristic of themovable object 102 (e.g., as described with regard to FIG. 5 and FIG.12).

In some embodiments, the predefined user input includes (2458) selectioninput for selecting a target 106 to track (e.g., as described withregard to FIG. 12).

In some embodiments, altering the response to the predefined user inputincludes (2460) ceasing to respond to the predefined user input.

In some embodiments, the device determines (2462) an updated location ofthe obstacle 1316. In some embodiments, the device determines (2464)whether the updated location of the obstacle 1316 corresponds to thereactive region 1602 relative to the movable object 102. In someembodiments, in response to determining that the updated location of theobstacle 1316 does not correspond to the reactive region 1602 relativeto the movable object 102, the device further adjusts (2466) the targettracking information and sends the further adjusted target trackinginformation to the control unit 104. In some embodiments, the furtheradjusted target tracking information includes information for ceasing toalter the response to the predefined user input received by the controlunit 104. For example, when the movable object 102 is no longer at riskof (imminent) collision with obstacle 1316, user ability to controlmovable object 102 using control device 104 resumes.

In some embodiments, the current location information of the obstacle1316 includes (2468) information obtained using one or more depth maps.

In some embodiments, the current location information of the obstacle1316 includes (2470) information obtained using one or more sensors ofmovable object sensing system 122.

In some embodiments, the reactive region 1602 is defined (2472) at leastin part based on a determined distance (e.g. 1604) from the movableobject 102. In some embodiments, the determined distance is based on oneor more current movement characteristics of the movable object 102.

In some embodiments, the determined distance is further based on (2474)one or more current movement characteristics of the obstacle 1316.

FIGS. 25A-25G are a flow diagram illustrating a method 2500 forcontrolling a movable object 102, in accordance with some embodiments.The method 2500 is performed at a device, such as moving object 102,control unit 104 and/or computing device 126. For example, instructionsfor performing the method 2500 are stored in motion control module 402of memory 118 and executed by processor(s) 116.

The device obtains (2502) current location information of an obstacle1316 while the movable object 102 tracks a target 106.

The device generates (2504) a plurality of sets of candidate movementcharacteristics (e.g., sets 2002, 2004, 2006 of plot 2000; sets 2102,2104, 2106 of plot 2001) for the movable object 102 based on the currentlocation information of the obstacle 1316 and a set of current movementcharacteristics of the movable object 102.

The device selects (2506), from the plurality of sets of candidatemovement characteristics for the movable object 102, a set of movementcharacteristics for the movable object 102.

The device adjusts (2508) one or more movement characteristics of themovable object 102 based on the selected set of movement characteristicsfor the movable object 102.

In some embodiments, after adjusting (2510) the one or more movementcharacteristics of the movable object 102 based on the selected set ofmovement characteristics for the movable object 102 (e.g., afteradjusting the one or more movement characteristics for a predefined timeperiod, such as 1 second), the device repeats the obtaining, generating,selecting, and adjusting operations.

In some embodiments, the device assigns (2512) a route optimizationscore to each set of candidate movement characteristics of the pluralityof sets of candidate movement characteristics and determines a set ofmovement characteristics for the movable object 102 that has a highestroute optimization score. In some embodiments, selecting the set ofmovement characteristics for the movable object 102 includes selectingthe set of candidate movement characteristics for the movable object 102that has the highest route optimization score.

In some embodiments, after adjusting (2514) the one or more movementcharacteristics of the movable object 102 based on the selected set ofmovement characteristics for the movable object 102 (e.g., afteradjusting the one or more movement characteristics for a predefined timeperiod, such as 1 second), the device repeats the obtaining, generating,assigning, determining, selecting, and adjusting operations.

In some embodiments, the device predicts (2516), for a set of candidatemovement characteristics, a time at which the movable object 102 willcollide with the obstacle 1316 and determines a route optimization scoreto assign to the set of candidate movement characteristics based atleast in part on a difference between a current time and the predictedtime at which the movable object 102 will collide with the obstacle1316.

In some embodiments, the device determines (2518), for a set ofcandidate movement characteristics, whether the movable object 102 willcollide with the obstacle 1316, in response to determining that themovable object 102 will collide with the obstacle 1316, the devicepredicts, for the set of candidate movement characteristics, a time atwhich the movable object 102 will collide with the obstacle 1316, anddetermines a route optimization score to assign to the set of candidatemovement characteristics based at least in part on a difference betweena current time and the predicted time at which the movable object 102will collide with the obstacle 1316. In response to determining that themovable object 102 will not collide with the obstacle 1316, the devicedetermines a route optimization score to assign to the set of candidatemovement characteristics based at least in part on a default valuecorresponding to a determination that the movable object 102 will notcollide with the obstacle 1316.

In some embodiments, the predicted amount of time before the movableobject 102 will collide with the obstacle 1316 is determined (2520)using one or more current movement characteristics of the obstacle 1316.

In some embodiments, the current location information of the obstacle1316 includes (2522) the one or more current movement characteristics ofthe obstacle 1316.

In some embodiments, the one or more current movement characteristics ofthe obstacle 1316 are determined (2524) using one or more sensors ofmovable object sensing system 122.

In some embodiments, the device determines (2526) a route optimizationscore to assign to a respective set of candidate movementcharacteristics based at least in part on differences between the set ofcandidate movement characteristics and the set of current movementcharacteristics of the movable object 102.

In some embodiments, the device predicts (2528), for a set of candidatemovement characteristics, a distance between the movable object 102 andthe target 106 at a predetermined future time; and the device determinesa route optimization score to assign to the set of candidate movementcharacteristics based at least in part on the predicted distance betweenthe movable object 102 and the target 106 at the predetermined futuretime.

In some embodiments, predicting the distance between the movable object102 and the target 106 at a predetermined future time includes (2530)obtaining at least one current movement characteristic of the target106. The device predicts the distance using the obtained at least onecurrent movement characteristic of the target 106.

In some embodiments, the device assigns a route optimization score toeach set of candidate movement characteristics of the plurality of setsof candidate movement characteristics. The device determines a set ofmovement characteristics for the movable object 102 that has a highestroute optimization score. The device determines whether the set ofmovement characteristics for the movable object 102 that has the highestroute optimization score complies with (2532) constraint criteria. Inresponse to determining that the set of candidate movementcharacteristics complies with the constraint criteria, the deviceselects the set of movement characteristics for the movable object 102that has the highest route optimization score. In response todetermining that the set of candidate movement characteristics does notcomply with the constraint criteria, the device selects an alternativeset of movement characteristics for the movable object 102.

For example, in some embodiments, one or more movement characteristicsof movable object 102, such as a linear velocity, an angular velocity, alinear acceleration, an angular acceleration, and/or an altitude areconstrained. For example, a constraint exists due to, e.g., a mechanicallimit (e.g., a mechanical limit of an actuator controlling a movementmechanism 114) and/or a policy limit (e.g., a law limiting allowablevelocity, acceleration, and/or elevation). In some embodiments,selection of a set of candidate movement characteristics and/orgeneration of candidate movement characteristics is constrained based onthe constraints.

In some embodiments, the constraint criteria include (2534) a maximumlinear velocity of the movable object 102.

In some embodiments, the constraint criteria include a (2536) maximumangular velocity of the movable object 102.

In some embodiments, the plurality of sets of candidate movementcharacteristics are generated (2538) subject to constraint criteria.

In some embodiments, the constraint criteria include (2540) a maximumlinear velocity of the movable object 102.

In some embodiments, the constraint criteria include (2542) a maximumangular velocity of the movable object 102.

In some embodiments, a respective set of candidate movementcharacteristics includes (2544) a y-axis movement characteristic and az-axis movement characteristic (e.g., V_(Z)). A movement characteristicis, e.g., an x-axis velocity V_(X), an x-axis acceleration, a y-axisvelocity V_(Y), a y-axis acceleration, a z-axis velocity V_(Z), and/orz-axis acceleration.

In some embodiments, the y-axis movement characteristic and the z-axismovement characteristic are determined (2546) in a frame of reference1900 of the movable object 102 (e.g., as described with regard to FIG.19).

In some embodiments, the respective set of candidate movementcharacteristics includes (2548) a y-axis velocity V_(Y) and a z-axisvelocity V_(Z).

In some embodiments, the plurality of sets of candidate movementcharacteristics include (2550): a respective set of candidate movementcharacteristics including a negative y-axis velocity value and anegative z-axis velocity value; a set of candidate movementcharacteristics including a negative y-axis velocity value and apositive z-axis velocity value; a set of candidate movementcharacteristics including a positive y-axis velocity value and anegative z-axis velocity value; and a set of candidate movementcharacteristics including a positive y-axis velocity value and apositive z-axis velocity value. For example, candidate movementcharacteristics in FIG. 20 are shown in each quadrant of the coordinateplot 2000.

In some embodiments, a respective set of candidate movementcharacteristics includes (2552) an x-axis movement characteristic and anangular velocity (e.g., ox). An x-axis movement characteristic is, e.g.,an x-axis velocity V_(X) and/or an x-axis acceleration.

In some embodiments, the x-axis movement characteristic and the angularvelocity are determined (2554) in a frame of reference 1900 of themovable object 102.

In some embodiments, the x-axis movement characteristic (2556) is avelocity V_(X).

In some embodiments, the plurality of sets of candidate movementcharacteristics include (2558): a set of candidate movementcharacteristics including a negative x-axis velocity value and anegative angular velocity value; a set of candidate movementcharacteristics including a negative x-axis velocity value and apositive angular velocity value; a set of candidate movementcharacteristics including a positive x-axis velocity value and anegative angular velocity value; and a set of candidate movementcharacteristics including a positive x-axis velocity value and apositive angular velocity value. For example, candidate movementcharacteristics in FIG. 22 are shown in each quadrant of the coordinateplot 2100.

In some embodiments, the device determines (2560) a size of the obstacle1316. The device determines whether the size of the obstacle meets firstobstacle size criteria. In response to determining that the size of theobstacle meets first obstacle size criteria, the multiple sets ofcandidate movement characteristics include movement of the movableobject along a y-axis and a z-axis relative to the movable object. Inresponse to determining that the size of the obstacle does not meetfirst obstacle size criteria, a respective set of candidate movementcharacteristics of the multiple sets of candidate movementcharacteristics include movement of the movable object along an x-axisrelative to the movable object, and angular velocity of the movableobject.

Many features of the present disclosure can be performed in, using, orwith the assistance of hardware, software, firmware, or combinationsthereof. Consequently, features of the present disclosure may beimplemented using a processing system. Exemplary processing systems(e.g., processor(s) 116, controller 210, controller 218, processor(s)502 and/or processor(s) 602) include, without limitation, one or moregeneral purpose microprocessors (for example, single or multi-coreprocessors), application-specific integrated circuits,application-specific instruction-set processors, field-programmable gatearrays, graphics processing units, physics processing units, digitalsignal processing units, coprocessors, network processing units, audioprocessing units, encryption processing units, and the like.

Features of the present disclosure can be implemented in, using, or withthe assistance of a computer program product, such as a storage medium(media) or computer readable medium (media) having instructions storedthereon/in which can be used to program a processing system to performany of the features presented herein. The storage medium (e.g., (e.g.memory 118, 504, 604) can include, but is not limited to, any type ofdisk including floppy disks, optical discs, DVD, CD-ROMs, microdrive,and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs,DDR RAMs, flash memory devices, magnetic or optical cards, nanosystems(including molecular memory ICs), or any type of media or devicesuitable for storing instructions and/or data.

Stored on any one of the machine readable medium (media), features ofthe present disclosure can be incorporated in software and/or firmwarefor controlling the hardware of a processing system, and for enabling aprocessing system to interact with other mechanism utilizing the resultsof the present disclosure. Such software or firmware may include, but isnot limited to, application code, device drivers, operating systems andexecution environments/containers.

Communication systems as referred to herein (e.g., communication systems120, 510, 610) optionally communicate via wired and/or wirelesscommunication connections. For example, communication systems optionallyreceive and send RF signals, also called electromagnetic signals. RFcircuitry of the communication systems convert electrical signalsto/from electromagnetic signals and communicate with communicationsnetworks and other communications devices via the electromagneticsignals. RF circuitry optionally includes well-known circuitry forperforming these functions, including but not limited to an antennasystem, an RF transceiver, one or more amplifiers, a tuner, one or moreoscillators, a digital signal processor, a CODEC chipset, a subscriberidentity module (SIM) card, memory, and so forth. Communication systemsoptionally communicate with networks, such as the Internet, alsoreferred to as the World Wide Web (WWW), an intranet and/or a wirelessnetwork, such as a cellular telephone network, a wireless local areanetwork (LAN) and/or a metropolitan area network (MAN), and otherdevices by wireless communication. Wireless communication connectionsoptionally use any of a plurality of communications standards, protocolsand technologies, including but not limited to Global System for MobileCommunications (GSM), Enhanced Data GSM Environment (EDGE), high-speeddownlink packet access (HSDPA), high-speed uplink packet access (HSUPA),Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA),long term evolution (LTE), near field communication (NFC), wideband codedivision multiple access (W-CDMA), code division multiple access (CDMA),time division multiple access (TDMA), Bluetooth, Wireless Fidelity(Wi-Fi) (e.g., IEEE 102.11a, IEEE 102.11ac, IEEE 102.11ax, IEEE 102.11b,IEEE 102.11g and/or IEEE 102.11n), voice over Internet Protocol (VoIP),Wi-MAX, a protocol for e-mail (e.g., Internet message access protocol(IMAP) and/or post office protocol (POP)), instant messaging (e.g.,extensible messaging and presence protocol (XMPP), Session InitiationProtocol for Instant Messaging and Presence Leveraging Extensions(SIMPLE), Instant Messaging and Presence Service (IMPS)), and/or ShortMessage Service (SMS), or any other suitable communication protocol,including communication protocols not yet developed as of the filingdate of this document.

While various embodiments of the present disclosure have been describedabove, it should be understood that they have been presented by way ofexample, and not limitation. It will be apparent to persons skilled inthe relevant art that various changes in form and detail can be madetherein without departing from the spirit and scope of the disclosure.

The present disclosure has been described above with the aid offunctional building blocks illustrating the performance of specifiedfunctions and relationships thereof. The boundaries of these functionalbuilding blocks have often been arbitrarily defined herein for theconvenience of the description. Alternate boundaries can be defined solong as the specified functions and relationships thereof areappropriately performed. Any such alternate boundaries are thus withinthe scope and spirit of the disclosure.

The terminology used in the description of the various describedembodiments herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used in thedescription of the various described embodiments and the appendedclaims, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

As used herein, the term “if” may be construed to mean “when” or “upon”or “in response to determining” or “in accordance with a determination”or “in response to detecting,” that a stated condition precedent istrue, depending on the context. Similarly, the phrase “if it isdetermined [that a stated condition precedent is true]” or “if [a statedcondition precedent is true]” or “when [a stated condition precedent istrue]” may be construed to mean “upon determining” or “in response todetermining” or “in accordance with a determination” or “upon detecting”or “in response to detecting” that the stated condition precedent istrue, depending on the context.

The foregoing description of the present disclosure has been providedfor the purposes of illustration and description. It is not intended tobe exhaustive or to limit the disclosure to the precise forms disclosed.The breadth and scope of the present disclosure should not be limited byany of the above-described exemplary embodiments. Many modifications andvariations will be apparent to the practitioner skilled in the art. Themodifications and variations include any relevant combination of thedisclosed features. The embodiments were chosen and described in orderto best explain the principles of the disclosure and its practicalapplication, thereby enabling others skilled in the art to understandthe disclosure for various embodiments and with various modificationsthat are suited to the particular use contemplated. It is intended thatthe scope of the invention be defined by the following claims and theirequivalence.

What is claimed is:
 1. A method for controlling a movable object,comprising: obtaining current location information of an obstacle whilethe movable object tracks a target; determining, based on the currentlocation information of the obstacle, whether a location of the obstaclecorresponds to a reactive region relative to the movable object; inresponse to determining that the location of the obstacle corresponds tothe reactive region, adjusting one or more movement characteristics ofthe movable object in a reactive manner to prevent the movable objectfrom colliding with the obstacle; and in response to determining thatthe location of the obstacle does not correspond to the reactive region,adjusting the one or more movement characteristics of the movable objectin a proactive manner to maintain a distance between-the movable objectand the obstacle to be larger than a predefined distance, whereinadjusting the one or more movement characteristics of the movable objectin the proactive manner includes: selecting multiple sets of candidatemovement characteristics based on the one or more movementcharacteristics of the movable object; obtaining a route optimizationscore for each set of candidate movement characteristics among themultiple sets of candidate movement characteristics; determining aselected set of candidate movement characteristics having a highestroute optimization score; and adjusting the one or more movementcharacteristics of the movable object based on the selected set ofcandidate movement characteristics.
 2. The method of claim 1, whereinthe current location information of the obstacle includes informationobtained using one or more depth maps, each of the one or more depthmaps being obtained using at least one sensor of the movable object. 3.The method of claim 2, wherein the at least one sensor of the movableobject includes a pair of sensors for stereoscopic depth mapping.
 4. Themethod of claim 1, wherein the current location information of theobstacle includes at least one of a position of the obstacle or one ormore movement characteristics of the obstacle.
 5. The method of claim 1,wherein the current location information of the obstacle is transmittedfrom a computing device to the movable object or received by the movableobject from the computing device.
 6. The method of claim 1, wherein: thereactive region is defined at least in part based on a determineddistance from the movable object; and the determined distance is basedon at least one of one or more current movement characteristics of themovable object or one or more current movement characteristics of theobstacle.
 7. The method of claim 1, wherein obtaining the routeoptimization score for the each set of candidate movementcharacteristics includes: predicting, for the each set of candidatemovement characteristics, a time at which the movable object willcollide with the obstacle using one or more movement characteristics ofthe obstacle; and determining the route optimization score for the eachset of candidate movement characteristics based on a difference betweena current time and the predicted time at which the movable object willcollide with the obstacle.
 8. The method of claim 1, wherein the routeoptimization score for a set of candidate movement characteristicsdepends, at least in part, upon differences between the set of candidatemovement characteristics and a set of current movement characteristicsof the movable object.
 9. The method of claim 1, wherein adjusting theone or more movement characteristics of the movable object in theproactive manner further includes: predicting, for a set of candidatemovement characteristics, a distance between the movable object and thetarget at a predetermined future time; wherein the route optimizationscore for the set of candidate movement characteristics depends, atleast in part, upon the predicted distance between the movable objectand the target at the predetermined future time.
 10. The method of claim1, wherein a respective set of candidate movement characteristicsincludes: a first movement characteristic in a first direction that isperpendicular to movement of the movable object and a second movementcharacteristic in a second direction that is perpendicular to movementof the movable object and distinct from the first direction; or amovement characteristic in a direction of movement of the movable objectand an angular velocity.
 11. The method of claim 10, wherein thedirection of movement of the movable object is along an x-axisperpendicular to a y-axis and a z-axis, and the method furthercomprises: determining a size of the obstacle; and determining whetherthe size of the obstacle meets obstacle size criteria, wherein: inresponse to determining that the size of the obstacle meets the obstaclesize criteria, the multiple sets of candidate movement characteristicsinclude movement characteristics of the movable object along the y-axisand the z-axis relative to the movable object, and in response todetermining that the size of the obstacle does not meet the obstaclesize criteria, a respective set of candidate movement characteristics ofthe multiple sets of candidate movement characteristics include: themovement characteristic of the movable object along the x-axis relativeto the movable object, and the angular velocity of the movable object.12. The method of claim 1, wherein adjusting the one or more movementcharacteristics in the reactive manner further includes: determining,based on the current location information of the obstacle, whether thelocation of the obstacle corresponds to a first sub-region or a secondsub-region of the reactive region relative to the movable object, anarea of the second sub-region of the reactive region relative to themovable object being smaller than an area of the first sub-region of thereactive region relative to the movable object; in response todetermining that the location of the obstacle corresponds to the firstsub-region of the reactive region relative to the movable object,reducing an acceleration of the movable object; and in response todetermining that the location of the obstacle corresponds to the secondsub-region of the reactive region relative to the movable object,reducing a velocity of the movable object.
 13. The method of claim 12,wherein adjusting the one or more movement characteristics in thereactive manner further includes: determining, based on the currentlocation information of the obstacle, whether the location of theobstacle corresponds to a third sub-region of the reactive regionrelative to the movable object, an area of the third sub-region of thereactive region relative to the movable object being smaller than thearea of the second sub-region of the reactive region relative to themovable object; and in response to determining that the location of theobstacle corresponds to the third sub-region of the reactive regionrelative to the movable object, reversing a direction of movement of themovable object.
 14. The method of claim 1, further comprising: inresponse to determining that the location of the obstacle corresponds tothe reactive region, transmitting adjusted target tracking informationto a control unit.
 15. The method of claim 14, wherein the adjustedtarget tracking information includes information about the obstacle. 16.A system for controlling a movable object, comprising: one or moreprocessors; and a memory storing one or more programs configured to beexecuted by the one or more processors to: obtain current locationinformation of an obstacle while the movable object tracks a target;determine, based on the current location information of the obstacle,whether a location of the obstacle corresponds to a reactive regionrelative to the movable object; in response to determining that thelocation of the obstacle corresponds to the reactive region, adjust oneor more movement characteristics of the movable object in a reactivemanner to prevent the movable object from colliding with the obstacle;and in response to determining that the location of the obstacle doesnot correspond to the reactive region, adjust the one or more movementcharacteristics of the movable object in a proactive manner to maintaina distance between the movable object and the obstacle to be larger thana predefined distance, wherein adjusting the one or more movementcharacteristics of the movable object in the proactive manner includes:selecting multiple sets of candidate movement characteristics based onthe one or more movement characteristics of the movable object;obtaining a route optimization score for each set of candidate movementcharacteristics among the multiple sets of candidate movementcharacteristics; determining a selected set of candidate movementcharacteristics having a highest route optimization score; and adjustingthe one or more movement characteristics of the movable object based onthe selected set of candidate movement characteristics.
 17. An unmannedaerial vehicle (UAV), comprising: a propulsion system; one or moresensors; and one or more processors individually or collectivelyconfigured to: obtain, using the one or more sensors, current locationinformation of an obstacle while the UAV tracks a target; determine,based on the current location information of the obstacle, whether alocation of the obstacle corresponds to a reactive region relative tothe UAV; in response to determining that the location of the obstaclecorresponds to the reactive region, adjust one or more movementcharacteristics of the UAV in a reactive manner to prevent the UAV fromcolliding with the obstacle is avoided; and in response to determiningthat the location of the obstacle does not correspond to the reactiveregion, adjust one or more movement characteristics of the UAV in aproactive manner to maintain a distance between the UAV and the obstacleto be larger than a predefined distance, wherein adjusting the one ormore movement characteristics of the UAV in the proactive mannerincludes: selecting multiple sets of candidate movement characteristicsbased on the one or more movement characteristics of the UAV; obtaininga route optimization score for each set of candidate movementcharacteristics among the multiple sets of candidate movementcharacteristics; determining a selected set of candidate movementcharacteristics having a highest route optimization score; and adjustingthe one or more movement characteristics of the UAV based on theselected set of candidate movement characteristics.
 18. The UAV of claim17, wherein the one or more sensors include a pair of sensors forstereoscopic depth mapping.
 19. The UAV of claim 18, wherein the one ormore processors are further configured to, individually or collectively,use a depth map to determine the current location information of theobstacle.